How to index in elasticsearch using post request in Python?

Question

I am facing an issue:

RequestError(400, 'illegal_argument_exception', 'mapper [columns.analysis.abstract_stats.description.std] of different type, current_type [text], merged_type [float]')

which led me to go for a solution described here.

My current code which is generating the aforementioned error is:

from test_mapping import a

es = Elasticsearch([{'host': 'A.B.C.D', 'port': 9200}])

try:
    es.index(index='datatables', doc_type='datatable_v1', id="pallet_d3dd6729b810bebd955708e85afc1f65c3f2685c", body=a)
except Exception as e:
    print (e)

The index existed before but I have deleted it and then running the above code is still generating the above error. The variable a is here

can you share the mapping of the doc type datatable_v1 you have used? — Aman Bansal
– Aman Bansal, Commented Nov 30, 2018 at 9:25
also there is problem with your data too. Many documents have this columns.analysis.abstract_stats.description.std set to 'Nan'. This is letting the ES to dynamically initialize the field as text — Aman Bansal
– Aman Bansal, Commented Nov 30, 2018 at 9:29

Shanteshwar Inde · Accepted Answer · 2018-11-30 12:15:23Z

1

Use this to load a:

import simplejson
es.index(index='datatables', doc_type = 'datatable_v1', id = "pallet_d3dd6729b810bebd955708e85afc1f65c3f2685c", body = simplejson.dumps(a, ignore_nan = True))

This should solve your problem. Now your application will read this value as None (which is possibly the source of this corruption) and you can easily implement your functionality

edited Nov 30, 2018 at 12:15

Shanteshwar Inde

1,4564 gold badges19 silver badges29 bronze badges

answered Nov 30, 2018 at 11:42

Aman Bansal

915 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Nishant · Accepted Answer · 2018-12-01 12:08:13Z

1

The reason for the above error is when you send data to elastic then it created dynamic field for the keys it find missing in the mapping and try to identify its type. Base on the data you are sending in body the value at columns.analysis.abstract_stats.description.std is mapped to float type but one of the record at key columns.analysis.abstract_stats.description.std has value 'NaN' which can't be mapped to a float field and hence the error. You need to make sure that type of fields doesn't change from one record to another.

edited Dec 1, 2018 at 12:08

answered Nov 30, 2018 at 10:00

Nishant

7,9121 gold badge24 silver badges37 bronze badges

Collectives™ on Stack Overflow

How to index in elasticsearch using post request in Python?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related