elasticsearch partial update with python

Question

I have a elasticsearch document in the following format. I need to partial update the "x" field and add a python dict in it.

{
        "_index": "gdata34",
        "_type": "gdat",
        "_id": "328091-72341-118",
        "_version": 1,
        "_score": 1,
        "_source": {
            "d": {
                "Thursday": {
                    "s": ""
                },
                "Email": {
                    "s": ""
                },
               "Country": {
                    "s": "US"
                },

            },
            "x": {
                "Geo": {
                    "s": "45.335428,-118.057133",
                    "g": [
                        -118.057133
                        ,
                        45.335428
                    ]
                }
            },
          }
        }

I tried the following code to update:

from elasticsearch import Elasticsearch, exceptions
import pprint


elasticsearch = Elasticsearch()
doc = elasticsearch.get(index='gdata34', doc_type='gdat', id='328091-72341-7')

elasticsearch.update(index='gdata34', doc_type='gdat', id='328091-72341-7',
                     body={"script":"ctx._source.x += y",
                           "params":{"y":"z"}
                     }
                     )
elasticsearch.indices.refresh(index='gdata34')
new_doc = elasticsearch.get(index='gdata34', doc_type='gdat', id='328091-72341-7')

I am getting this error:

elasticsearch.exceptions.RequestError: TransportError(400, u'ElasticsearchIllegalArgumentException[failed to execute script]; nested: ScriptException[dynamic scripting for [groovy] disabled]; ')

What is the right way to do partial update in elasticsearch using python?

What version of ES are you using?

Lukas Graf
– Lukas Graf

2015-04-06 10:42:05 +00:00
Commented Apr 6, 2015 at 10:42 — Lukas Graf
– Lukas Graf, Commented Apr 6, 2015 at 10:42

Anish · Accepted Answer · 2015-04-06 10:55:03Z

13

For future reference, the following method of partial update worked.

elasticsearch.update(index='gdata34', doc_type='gdat', id='328091-72341-7',
                     body={
                         'doc': {'x': {'y':'z'}}
                     }
                     )

answered Apr 6, 2015 at 10:55

Anish

1,97011 gold badges29 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lukas Graf · Accepted Answer · 2015-04-06 10:59:20Z

1

From the ElasticSearch docs on scripting:

We recommend running Elasticsearch behind an application or proxy, which protects Elasticsearch from the outside world. If users are allowed to run dynamic scripts (even in a search request), then they have the same access to your box as the user that Elasticsearch is running as. For this reason dynamic scripting is allowed only for sandboxed languages by default.

Now, in recent ES version there has been a bug in the vulnerability in the Groovy scripting engine that allows scripts to escape the sandbox and execute shell commands as the user running the Elasticsearch Java VM - that's why the Groovy sandbox is disabled by default in recent versions, and therefore the execution of Groovy scripts passed in the request body or from the .scripts index. The only way to execute Groovy scripts with this default configuration is to place them in the config/scripts/ directory on the node.

So you have two options:

If your ES instance is not directly accessible and is secured behind a proxy, you can turn on Groovy sandboxing again by setting script.groovy.sandbox.enabled: true in config/elasticsearch.yml on your node(s). If your ES instance is accessible by your
You can prepare your script and place it on the filesystem in the config/scripts directory of your node(s), and call it by name. See Running Groovy Scripts without Dynamic Scripting for details.

answered Apr 6, 2015 at 10:59

Lukas Graf

33k8 gold badges83 silver badges95 bronze badges

3 Comments

Anish Over a year ago

can we use the following?elasticsearch.update(index='gdata34', doc_type='gdat', id='328091-72341-7', body={ 'doc': {'x': {'y':'z'}} } )

Lukas Graf Over a year ago

Yes, but make sure to read the documentation - unless you specify "detect_noop": true this will always result in the document being updated, even if the merging process didn't detect any changes.

Ville Laurikari Over a year ago

At least with ElasticSearch 2.3 (the latest version at this time), detect_noop is enabled by default.

Collectives™ on Stack Overflow

elasticsearch partial update with python

2 Answers 2

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related