How to parse a single line json file containing multiple objects

Question

I need to read some JSON data for processing. I have a single line file that has multiple JSON objects how can I parse this?

I want the output to be a file with a single line per object.

I have tried a brute force method that will use json.loads recursively to check if the json is valid but I'm getting different results every time I run the program

import json

with open('sample.json') as inp:
s = inp.read()

jsons = []

start, end = s.find('{'), s.find('}')
while True:
 try:
    jsons.append(json.loads(s[start:end + 1]))
    print(jsons)
except ValueError:
    end = end + 1 + s[end + 1:].find('}')
else:
    s = s[end + 1:]
    if not s:
        break
    start, end = s.find('{'), s.find('}')

for x  in jsons:
  writeToFilee(x)

The json format can be seen here https://pastebin.com/DgbyjAG9

Paste a sample of your file along with how you'd like to have the output. — DirtyBit
– DirtyBit, Commented Apr 9, 2019 at 12:49
I want to split the single line file containing multiple objects to a multiple line file containing an object on each line — Faber
– Faber, Commented Apr 9, 2019 at 12:53
@Jessica are these objects delimited somehow? Or is it just like {...}{...}? I found only 1 occurrence of "}\s*{" regex in the paste you provided, am I right to assume this file contains 2 different JSON objects, or are there more? — mr_m1m3
– mr_m1m3, Commented Apr 9, 2019 at 13:00
how about jsons = s.replace('}{', '}|{').split('|') to create a list of json strings? — Chris Adams
– Chris Adams, Commented Apr 9, 2019 at 13:07

Sam Mason · Accepted Answer · 2019-04-09 13:28:08Z

4

why not just use the pos attribute of the JSONDecodeError to tell you where to delimit things?

something like:

import json

def json_load_all(buf):
    while True:
        try:
            yield json.loads(buf)
        except json.JSONDecodeError as err:
            yield json.loads(buf[:err.pos])
            buf = buf[err.pos:]
        else:
            break

works with your demo data as:

with open('data.json') as fd:
    arr = list(json_load_all(fd.read()))

gives me exactly two elements, but I presume you have more?

to complete this using the standard library, writing out would look something like:

with open('data.json') as inp, open('out.json', 'w') as out:
    for obj in json_load_all(inp.read()):
        json.dump(obj, out)
        print(file=out)

otherwise the jsonlines package is good for dealing with this data format

edited Apr 9, 2019 at 13:28

answered Apr 9, 2019 at 13:19

Sam Mason

16.5k1 gold badge49 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

C. Zona · Accepted Answer · 2019-04-09 13:41:22Z

1

The code below worked for me:

import json
with open(input_file_path) as f_in: 
    file_data = f_in.read() 
    file_data = file_data.replace("}{", "},{") 
    file_data = "[" + file_data + "]"
    data = json.loads(file_data)

answered Apr 9, 2019 at 13:41

C. Zona

1865 bronze badges

Comments

mr_m1m3 · Accepted Answer · 2019-04-09 13:37:00Z

0

Following @Chris A's comment, I've prepared this snippet which should work just fine:

with open('my_jsons.file') as file:
    json_string = file.read()

json_objects = re.sub('}\s*{', '}|!|{', json_string).split('|!|')
# replace |!| with whatever suits you best

for json_object in json_objects:
    print(json.loads(obj))

This example, however, will become worthless as soon as '}{' string appears in some value inside your JSON, so I strongly recommend using @Sam Mason's solution

edited Apr 9, 2019 at 13:37

answered Apr 9, 2019 at 13:30

mr_m1m3

3311 gold badge2 silver badges12 bronze badges

Collectives™ on Stack Overflow

How to parse a single line json file containing multiple objects

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related