Python JSON data importing

Question

I am trying to import data from a json file to sqlite but I got some error, I thought first that the problem is in the paths, I changed them in many ways but... no result, it gives me this error:

Traceback (most recent call last):
  File "C:/Users/Taner/PycharmProjects/untitled/MyProgram.py", line 13, in <module>
    json_object = json.loads(dummy_json)
  File "C:\Users\Taner\AppData\Local\Programs\Python\Python35-32\lib\json\__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

and here is my code:

import json
import bz2
import sqlite3

fpath = "C:\\Users\\Taner\\Downloads\\RC_2012-12.bz2"
databasepath = "C:\\Users\\Taner\\Desktop\\Seagate\\reddit"

conn = sqlite3.connect(databasepath)
curs = conn.cursor()
with bz2.BZ2File(fpath) as file:
    for line in file:
        dummy_json = line
        json_object = json.loads(dummy_json)
        po = json.loads(line.decode('utf8'))
        curs.execute("INSERT INTO Reddit VALUES (?,?,?)", (po['id'], po['subreddit_id'], po['subreddit'],))
        conn.commit()

oz123 · Accepted Answer · 2016-01-17 21:22:14Z

1

The correct form of your loop should be:

with bz2.BZ2File(fpath) as file:
    for line in file:
        po = json.loads(json.loads(line.decode('utf8')))
        curs.execute("INSERT INTO Reddit VALUES (?,?,?)", (po['id'], po['subreddit_id'], po['subreddit'],))
        conn.commit()

answered Jan 17, 2016 at 21:22

oz123

29.1k30 gold badges133 silver badges196 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Daniel Roseman · Accepted Answer · 2016-01-17 21:17:47Z

1

You already have the answer here, since in the line following the error you correctly decode the line from utf-8 before passing it to json.loads. The line with the error is therefore completely pointless, and you should remove it.

answered Jan 17, 2016 at 21:17

Daniel Roseman

602k68 gold badges911 silver badges924 bronze badges

Collectives™ on Stack Overflow

Python JSON data importing

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related