0

I'm building a json query to pass to a mongodb database in R.

In one scenario, I have a vector of dates and I want to query the database to return all records which have a date in the relevant field that matches a date in my vector of dates.

The second scenario is the same as the first, but this time I have a vector of character strings (IDs) and need to return all the records with matching IDs.

I understood the correct way to do this in a json query is to use the $in operator, and then put my vector in an array.

However, when I pass the query to my mongodb database, the exportLogId returns NULL. I'm quite sure that the problem is something to do with how I am representing the $in operator in the final query, since I have very similarly structured queries without the $in operator and they are all working. If I look for just one of my target dates or character strings, I get the desired result.

I followed the mongodb manual here to construct my query, and the only issue I can see is that the $in operator in the output of jsonlite::toJSON() is enclosed in double quotes; whereas I think it might need to be in single quotes (or no quotes at all, but I don't know how to write the syntax for that).

I'm creating my query in two steps:

  1. Create the query as a series of nested lists
  2. Convert the list object to json with jsonlite::toJSON()

Here is my code:

# Load libraries:
library(jsonlite)

# Create list of example dates to query in mongodb format:
sampledates <- c("2022-08-11T00:00:00.000Z",
                 "2022-08-15T00:00:00.000Z", 
                 "2022-08-16T00:00:00.000Z",
                 "2022-08-17T00:00:00.000Z", 
                 "2022-08-19T00:00:00.000Z")

# Create query as a list object:
    query_list_l <- list(filter =

                         # Add where clause:
                         list(where =

                                # Filter results by list of sample dates:
                                list(dateSampleTaken = list('$in' = sampledates),

                                     # Define format of column names and values:
                                     useDbColumns = "true",
                                     dontTranslateValues = "true",
                                     jsonReplaceUndefinedWithNull = "true"),

                              # Define columns to return:
                              fields = c("id",                
                                         "updatedAt",         
                                         "person.visualId",   
                                         "labName",           
                                         "sampleIdentifier",  
                                         "dateSampleTaken",   
                                         "sequence.hasSequence")))

# Convert list object to JSON:
query_json = jsonlite::toJSON(x = query_list_l, 
                              pretty = TRUE, 
                              auto_unbox = TRUE)

The JSON query now looks like this:

> query_json
{
  "filter": {
    "where": {
      "dateSampleTaken": {
        "$in": ["2022-08-11T00:00:00.000Z", "2022-08-15T00:00:00.000Z", "2022-08-16T00:00:00.000Z", "2022-08-17T00:00:00.000Z", "2022-08-19T00:00:00.000Z"]
      },
      "useDbColumns": "true",
      "dontTranslateValues": "true",
      "jsonReplaceUndefinedWithNull": "true"
    },
    "fields": ["id", "updatedAt", "person.visualId", "labName", "sampleIdentifier", "dateSampleTaken", "sequence.hasSequence"]
  }
} 

As you can see, $in is now enclosed in double quotes, even though I put it in single quotes when I created the query as a list object. I have tried replacing with sprintf() but that just adds a lot of backslashes to my query. I also tried:

query_fixed <- gsub(pattern = "\\"\\$\\in\\"", 
                    replacement = "\\'$in\\'", 
                    x = query_json)

... but this fails with an error.

I would be very grateful to know if:

  1. The syntax problem that is preventing $in from working is actually the double quotes?
  2. If double quotes is the problem, how do I replace them with single quotes without messing up the JSON format?

UPDATE:

The issue seems to occur when R is passing the query to the database, but I still can't work out exactly why.

If I try the query out in loopback explorer in the database, it works and using the export log ID produced, I can then fetch the results with httr::GET() in R. Example query results are shown below (sorry for the hashes - the main point is you can see the format of the returned values):

[1] "[{\"_id\":\"e59953b6-a106-4b69-9e25-1c54eef5264a\",\"updatedAt\":\"2022-09-12T20:08:39.554Z\",\"dateSampleTaken\":\"2022-08-16T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0044-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0002\"}},{\"_id\":\"af5cd9cc-4813-4194-b60b-7d130bae47bc\",\"updatedAt\":\"2022-09-12T20:11:07.467Z\",\"dateSampleTaken\":\"2022-08-17T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0061-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0003\"}},{\"_id\":\"b5930079-8d57-43a8-85c0-c95f7e0338d9\",\"updatedAt\":\"2022-09-12T20:13:54.378Z\",\"dateSampleTaken\":\"2022-08-16T00:00:00.000Z\",\"labName\":\"LNG_REFERENCE_DATA_CATEGORY_LAB_NAME_LAB_A\",\"sampleIdentifier\":\"LS0043-SCV2-PCR\",\"sequence\":{\"hasSequence\":false},\"person\":{\"visualId\":\"C-2022-0004\"}}]"
3
  • Do you have a sample document? In particular I'd like to confirm that the documents have string representations for their dateSampleTaken fields (as opposed to dates) since that is what this query is passing in Commented Oct 12, 2022 at 16:53
  • Yes I can confirm they are strings (at least when I pass a single date as a string in the same format, it retrieves matching records from the same document) Commented Oct 12, 2022 at 17:00
  • @user20042973 I updated my question with example output Commented Oct 12, 2022 at 18:21

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.