Incorrect file format to parse JSON in Python

Question

My file contains number of JSON records as follows:

Input file:

{"timestamp":1487271527,"user":"Dave","action":"browse"}
{"timestamp":1487271528,"user":"Dave","action":"navigate"}
{"timestamp":1487271529,"user":"Dave","action":"browse"}
{"timestamp":1487271530,"user":"Dave","action":"view"}
{"timestamp":1487271531,"user":"Dave","action":"browse"}
{"timestamp":1487271532,"user":"Dave","action":"browse"}
{"timestamp":1487271533,"user":"Dave","action":"browse"}
{"timestamp":1487271534,"user":"Dave","action":"navigate"}

I want to load this data into dictionary similar to what json.load function does

How can I do that ?

with json.load I am getting following error:

Traceback (most recent call last):
  File "C:/Users/lenovo/AppData/Local/Programs/Python/Python36-32/Granular.py", line 5, in <module>
    input_data = json.load(open(r"C:\Users\lenovo\Desktop\nlp\input.txt",'r'))
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36-32\lib\json\__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36-32\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36-32\lib\json\decoder.py", line 342, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 57)

why don't you use json.load if that is the result you are trying to achieve? — user2969402
– user2969402, Commented Feb 2, 2018 at 4:30
getting error while using json.load json.load expects data in following format [{"timestamp":1487271527,"user":"Dave","action":"browse"}, {"timestamp":1487271528,"user":"Dave","action":"navigate"},..] — Nupur Kulkarni
– Nupur Kulkarni, Commented Feb 2, 2018 at 4:34
ok, you just are missing commas in your input. give me a sec — user2969402
– user2969402, Commented Feb 2, 2018 at 4:35

TomDotTom · Accepted Answer · 2018-02-02 06:29:17Z

1

Each line in your example file is a separate JSON structure. You probably want to make this clear in the extension of your filename, for example you could use lsjson to stand for line separated JSON.

In this case, you have to read each one in as a string and the unmarshal to a python dict:

import json

with open('example.lsjson') as fh:
    data = [json.loads(line) for line in fh.readlines()]

You'll end up with a list of dicts

from pprint import pprint
pprint(data)
[{u'action': u'browse', u'timestamp': 1487271527, u'user': u'Dave'},
 {u'action': u'navigate', u'timestamp': 1487271528, u'user': u'Dave'},
 {u'action': u'browse', u'timestamp': 1487271529, u'user': u'Dave'},
 {u'action': u'view', u'timestamp': 1487271530, u'user': u'Dave'},
 {u'action': u'browse', u'timestamp': 1487271531, u'user': u'Dave'},
 {u'action': u'browse', u'timestamp': 1487271532, u'user': u'Dave'},
 {u'action': u'browse', u'timestamp': 1487271533, u'user': u'Dave'},
 {u'action': u'navigate', u'timestamp': 1487271534, u'user': u'Dave'}]

answered Feb 2, 2018 at 6:29

TomDotTom

6,8444 gold badges44 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

LhasaDad Over a year ago

did not get a chance to get my solution in before TomDotTom did but this is the one I would have headed toward providing :-)

user2969402 · Accepted Answer · 2018-02-02 04:39:39Z

0

Your file is missing commas and the brackets for it to be read as an array by json.load. Just load the file as a string first and add the proper punctuation:

with open('test.json', 'r') as fp:
    document = fp.read()
document = '[' + document.replace('\n', ',\n') + ']'

Then write the result to another file:

with open('test2.json', 'w') as fp:
    fp.write(document)

And you will be able to open it with json.load:

import json

with open('test2.json', 'r') as fp:
    content = json.load(fp)

answered Feb 2, 2018 at 4:39

user2969402

1,2613 gold badges16 silver badges26 bronze badges

Comments

Rakesh · Accepted Answer · 2018-02-02 06:20:38Z

Just read the file as a normal text file and the convert each line to a JSON object using the json.loads method and append it to a list.

EX:

import json
path = "PATH/TO/INPUT_FILE/data.json"

data = []
with open(path, "r") as infile:
    for line in infile.readlines():
        data.append(json.loads(line))

print data

Output:

[{u'action': u'browse', u'timestamp': 1487271527, u'user': u'Dave'}, {u'action': u'navigate', u'timestamp': 1487271528, u'user': u'Dave'}, {u'action': u'browse', u'timestamp': 1487271529, u'user': u'Dave'}, {u'action': u'view', u'timestamp': 1487271530, u'user': u'Dave'}, {u'action': u'browse', u'timestamp': 1487271531, u'user': u'Dave'}, {u'action': u'browse', u'timestamp': 1487271532, u'user': u'Dave'}, {u'action': u'browse', u'timestamp': 1487271533, u'user': u'Dave'}, {u'action': u'navigate', u'timestamp': 1487271534, u'user': u'Dave'}]

Collectives™ on Stack Overflow

Incorrect file format to parse JSON in Python

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related