how can I parse json with a single line python command?

Question

I would like to use python to parse JSON in batch scripts, for example:

HOSTNAME=$(curl -s "$HOST" | python ?)

Where the JSON output from curl looks like:

'{"hostname":"test","domainname":"example.com"}'

How can I do this with a single line python command?

Much easier to use something like jq from the command line, which is dedicated to parsing JSON and so has a cleaner design: HOSTNAME=$(curl -s $HOST | jq -r '.hostname'). — chepner
– chepner, Commented Jul 20, 2016 at 17:39
Down voters, please state why you are down voting. It's difficult to improve questions if you don't know what's wrong with them. — Chris Snow
– Chris Snow, Commented Jul 20, 2016 at 17:41
@chepner true, but I don't have and can't install jq on the host in question, but I do have python and the json module. — Chris Snow
– Chris Snow, Commented Jul 20, 2016 at 17:43
For what purpose do you require the Python code to be "a single line"? — Jongware
– Jongware, Commented Jul 23, 2016 at 10:01
It's much easier to work with single line commands in a batch script. Multi line Python statements that require Python indentation are difficult to integrate into scripts using pipes and redirects without having to put the Python statements into a separate file as a Python script. — Chris Snow
– Chris Snow, Commented Jul 23, 2016 at 10:13

Mathias Bynens · Accepted Answer · 2022-07-06 09:31:07Z

23

Based on the JSON below being returned from the curl command ...

'{"hostname":"test","domainname":"example.com"}'

You can then use python to extract the hostname using the python json module:

HOSTNAME=$(curl -s "$HOST" |
  python -c \
    'import json,sys;print(json.load(sys.stdin)["hostname"])')

Note that I have split the line using a \ to make it more readable on stackoverflow. I've also simplified the command based on chepner's comment.

Original source: Parsing JSON with Unix tools

See also: https://wiki.python.org/moin/Powerful%20Python%20One-Liners

edited Jul 6, 2022 at 9:31

Mathias Bynens

151k54 gold badges224 silver badges254 bronze badges

answered Jul 20, 2016 at 17:36

Chris Snow

24.9k38 gold badges158 silver badges332 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

chepner Over a year ago

For a one-liner, I wouldn't bother with the intermediate variable obj: python -c 'import json,sys;print json.load(sys.stdin)["hostname"].

Xiao Over a year ago

For python3: python3 -c 'import json,sys; print(json.load(sys.stdin)["hostname"])'. Same thing but with parens for print

MarkHu Over a year ago

This answer, while technically correct, is not conveniently formatted with executable code snippets. For instance most people are more interested in parsing the JSON, than in making a curl call to a service running on particular host. (I tried to edit it for clarity, but was rejected.)

hustljian · Accepted Answer · 2018-01-10 08:31:45Z

18

echo '{"hostname":"test","domainname":"example.com"}' | python -m json.tool

answered Jan 10, 2018 at 8:31

hustljian

1,09513 silver badges9 bronze badges

2 Comments

Dan Locks Over a year ago

This only prettily prints the incoming json. There is no extraction of variables or array members.

Thilo Cestonaro Over a year ago

echo '{"hostname":"test","domainname":"example.com"}' | python -m json.tool | awk '/hostname/{ printf $2 }' | tr --delete '"'

tripleee · Accepted Answer · 2021-12-16 08:40:21Z

6

Since Python is multiplatform, is important to note differences between Linux and Windows, especially because of how they treat double-quotes/single-quotes differently.

Second, some previous answers are a little bit obsolete: in python2, print without parentheses was permitted. Nevertheless, in python3, print must be between parentheses.

Linux (bash)

It doesn't matter how you put double/single quotes. Json can be parsed in both ways with "keys" or 'keys'

HOSTNAME=$(curl -s "$HOST" |
  python3 -c 'import json,sys;print(json.load(sys.stdin)["hostname"])')

It also works: (pay attention at single/double quote at key)

HOSTNAME=$(curl -s "$HOST" |
  python3 -c "import json,sys;print(json.load(sys.stdin)['hostname'])")

Windows (powershell)

Keys in json MUST be between single quotes. Only the following syntax is accepted.
The ConvertTo-Json function generates object and works with keys between single quotes.

$HOSTNAME=(Invoke-RestMethod $HOST | `
  ConvertTo-Json | `
  python3 -c "import json,sys; print(json.load(sys.stdin)['hostname'])")

edited Dec 16, 2021 at 8:40

tripleee

192k37 gold badges318 silver badges369 bronze badges

answered Dec 3, 2021 at 11:32

Alejandro Galera

3,7213 gold badges27 silver badges43 bronze badges

1 Comment

Chris Snow Over a year ago

This is a very nice answer to my question. Up voted.

MonaPy · Accepted Answer · 2023-12-07 22:12:11Z

3

Run this:

$ python -m json.tool

It reads input from stdin and print out in a nicely. So with your questions:

HOSTNAME=$(curl -s "$HOST" | python -m json.tool )

answered Dec 7, 2023 at 22:12

MonaPy

4794 silver badges6 bronze badges

Collectives™ on Stack Overflow

how can I parse json with a single line python command?

4 Answers 4

3 Comments

2 Comments

Linux (bash)

Windows (powershell)

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

2 Comments

Linux (bash)

Windows (powershell)

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related