Parsing Json nested objects in Bash

Question

I'm accessing an API that returns JSON with nested objects like this:

{
  "9273": {
    "status": "ok",
    "tag": "group-8",
    "name": "London"
  },
  "4029": {
    "status": "unknown",
    "tag": "group-12",
    "name": "Tokyo"
  },
  "6322": {
    "status": "ok",
    "tag": "group-12",
    "name": "Singapore"
  },
  "1038": {
    "status": "unknown",
    "tag": "group-19",
    "name": "Melbourne"
  },
  "2938": {
    "status": "ok",
    "tag": "group-12",
    "name": "New York"
  }
}

I'm trying to parse the JSON using jq, regex, sed etc. in a Bash script, filter it on status (ok) and tag (highest group with an 'ok'), and flatten the matching objects to get a multi-line sorted string value of their name along with a static prefix (e.g. City).

Desired output below:

City: New York
City: Singapore

I'll appreciate any help in working this out.

peak · Accepted Answer · 2020-03-27 02:26:58Z

3

With your data, and using the -r command-line option, the following program produces the output shown below:

[.[]]
| map(select(.status == "ok"))
| (max_by( .tag | sub("group-";"") | tonumber) | .tag) as $mx
| .[]
| select(.tag==$mx)
| "City: \(.name)"

Output:

City: Singapore
City: New York

If you want the cities listed in alphabetical order, then you can add the appropriate sort_by filter to the jq pipeline.

answered Mar 27, 2020 at 2:26

peak

119k21 gold badges185 silver badges218 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Nick Over a year ago

Thanks, looks good. Where does the sort_by go in the chain?

Nick Over a year ago

Never mind. Figured out the sorting:

[.[]] | map(select(.status == "ok")) | (max_by( .tag | sub("group-";"") | tonumber) | .tag) as $mx | [.[]] | map(select(.tag==$mx)) | sort_by(.name) | .[] | "City: \(.name)"

Nick Over a year ago

By the way, I made a shorter alternative using group_by based on your answer:

[.[]] | map(select(.status == "ok")) | group_by(.tag) | max_by( .[].tag | sub("group-";"") | tonumber) | sort_by(.name)? | .[] | "City: \(.name)"

. Do you see any problem with it as such?

peak Over a year ago

Yes, there are some efficiency issues. Also, the semantics of max_by(STREAM) is a bit obscure.

Nick Over a year ago

Thanks for your time and review. So you reckon group_by is inefficient in this case?

Collectives™ on Stack Overflow

Parsing Json nested objects in Bash

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related