Search document with empty array field, on ElasticSearch

Question

I have a set of documents (type 'article') and I want to search for the document that have elements/objects into an array field

{
    "_type": "article",
    "_source": {
        "title": "Article 1",
        "locations": [
            {
                "address": "ES headquarter",
                "city": "Berlin"
            }
        ]
    }
}

I want two queries (just one, but with a little variation):

get all the articles that have locations
get all the articles that have NO locations

I tried different things but probably I'm too bad with ElasticSearch:

{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": [
        {
          "type": {
            "value": "article"
          }
        },
        {
          "bool": {
            "must_not": {
              "missing": {
                "field": "location",
                "existence": true,
                "null_value": true
              }
            }
          }
        }
      ]
    }
  }
}

this doesn't work.

How would you fix my query?

but mainly:

How would you perform this search for documents with a field that is an empty array?

Zoltan Balogh · Accepted Answer · 2014-11-06 14:15:04Z

16

If address is a mandatory field in location array you can modify your query:

"must_not": {
  "missing": {
    "field": "locations.address"
  }
}

AFAIK, in ES you cannot query non-leaf elements (like your location field) (see issue), and in case object types ES flattens the nested fields (see nested type, object type). That's why I suggested to query for one of the leaf elements instead. But it requires that one of them is mandatory (which is unfortunately not satisfied in your case).

Anyway I found the solution using the _source parameter inside the source_filtering:

"must_not": {
  "script": {
    "script": "_source.locations.size() > 0"
  }
}

Note that using "lang":"groovy" you should write: "script": "_source.locations.size > 0"

edited Nov 6, 2014 at 14:15

answered Nov 4, 2014 at 15:50

Zoltan Balogh

2783 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Kamafeather Over a year ago

No, it is not a mandatory field; how else could I solve?. And what would be the difference if it was mandatory?.

Kamafeather Over a year ago

Wow thanks! This seems to be almost the solution I need :) The problems:_________1) I get a script_lang not supported [groovy] (solved just removing the lang statement); and 2) I get a Error: could not access: size; in class: java.util.ArrayList and I don't know how to proceed with that :(

Zoltan Balogh Over a year ago

Ok, it might be beyond my actual knowledge of ES, but here is my tips:

Kamafeather Over a year ago

Solved! It was _source.locations.size() > 0! Missed the brackets! It works perfectly now :D! If you update your answer I'll set it as best one ^^

Zoltan Balogh Over a year ago

Super! Now it's clear: With mvel script language we should use size(), but with groovy we should use size.

|

eggplantkiller · Accepted Answer · 2018-09-18 16:29:11Z

15

If you don't want to enable scripting, you can combine the Exists Query with a must_not bool query, e.g:

{
  "query":{
    "bool":{
      "must_not":[
        {
          "exists":{
            "field":"tags"
          }
        }
      ]
    }
  }
}

answered Sep 18, 2018 at 16:29

eggplantkiller

1671 silver badge4 bronze badges

Comments

LazyCat01 · Accepted Answer · 2024-08-29 17:39:03Z

6

As per the ElasticSearch documentation

An empty array is treated as a missing field — a field with no values.

Let's suppose you have two documents in the article-index index

# First document
{
    "_type": "article",
    "_source": {
        "title": "Article 1",
        "locations": [{"address": "ES headquarter", "city": "Berlin"}]
    }
}

# Second document
{
    "_type": "article",
    "_source": {
        "title": "Article 2",
        "locations": []
    }
}

Expected queries would be:

Get all the articles that have locations

GET article-index/_search
{
  "query": {
    "exists": {
       "field": "locations"
    }
  }
}

get all the articles that have NO locations

GET article-index/_search
{
  "query": { 
    "bool": {
      "must_not": {
        "exists": {
          "field": "locations"
        }
      }
    }
  }
}

edited Aug 29, 2024 at 17:39

LazyCat01

1,0599 silver badges26 bronze badges

answered Jul 27, 2022 at 8:00

Azeem Chauhan

4677 silver badges8 bronze badges

1 Comment

472084 Over a year ago

I think you mean must_not on the last example?

Collectives™ on Stack Overflow

Search document with empty array field, on ElasticSearch

3 Answers 3

7 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related