MongoDB update data with script

Question

Is there a possibility to update data in mongodb with some sort of script? I don't want to (can't) access the mongo shell - but would like to perform the mongoshell update queries. My data is a csv file. I use hadoop for the analysis of the data (extraction and transformation). I need to get the data back in to mongodb and update some attributes. As reference for the update I would like to use the generated id

Can this task be done?

any help would be very appreciated

Phlip · Accepted Answer · 2015-08-19 18:23:55Z

15

You want to read data from a CSV file and import into mongodb? You could generate a script file (javascript) and use the mongo shell to execute it like described in "scripting the shell".

Example session, test database, starting with an empty foo collection:

> db.foo.insert({name : "james", position : "forward"})
> db.foo.find()
{ "_id" : ObjectId("4f0c99f6cb435f1e7d7fedea"), "name" : "james",     "position" : "forward" }
>

then you generate your script let's say mongo_scripting.js:

db.foo.insert({name : "wade", position : "guard"});
db.foo.update({name : "james"}, {$set : {position : "power forward"}}, false, true);

and running the script:

mongo localhost:27017/test  mongo_scripting.js

going back to mongo:

> db.foo.find()
{ "_id" : ObjectId("4f0c99f6cb435f1e7d7fedea"), "name" : "james", "position" : "power forward" }
{ "_id" : ObjectId("4f0c9a64a4a4642bae6459ea"), "name" : "wade",     "position" : "guard" }
>

you see that one document got updated and one new inserted.

An alternative is to use the java/python... driver to load the data.

edited Aug 19, 2015 at 18:23

Phlip

5,3505 gold badges39 silver badges52 bronze badges

answered Jan 10, 2012 at 16:49

milan

12.5k3 gold badges46 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Backlit Over a year ago

Hi,its not just an import. I want to update existing data. I looked at "scripting the shell". But its not much of help - or am I missing something?

Phlip Over a year ago

What's in the JavaScript file? Did I miss it? SEO-ing in, here!

A. Jesse Jiryu Davis · Accepted Answer · 2012-01-10 20:14:53Z

if you can connect to MongoDB at all, then you can surely use the shell. Just run the shell on your local machine and tell it to connect to the remote Mongo instance, like:

mongo dbserver.mydomain.com/foo

You can also consider using mongoimport, http://www.mongodb.org/display/DOCS/Import+Export+Tools , although mongoimport will want to create or replace whole documents, not update fields within documents as you've asked.

It sounds to me like you'll need to write a script to process each line of the CSV and update documents in MongoDB. In Python, that script would go something like:

import csv, pymongo, sys
foo_db = pymongo.Connection("dbserver.mydomain.com").foo
csv_reader = csv.reader(open(sys.argv[1], 'rb'), delimiter=',', quotechar='"')
for line in csv_reader:
    _id, field1, field2 = line
    foo_db.my_collection.update({
        "_id": _id
    }, {
        "$set": { "field1": field1, "field2": field2 }
    }, safe=True)

Collectives™ on Stack Overflow

MongoDB update data with script

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related