Extract values from JSON objects and assign them to shell variables in a loop

Question

I am using jq like below to parse a json file -

The json file looks something like -

{
  "values": [
    {
      "email": "[email protected]",
      "id": "USER1_ID"
    },{
      "email": "[email protected]",
      "id": "USER2_ID"
    },
    .
]
}

I am able to print/ iterate through the ids like below

for k in $(cat input.json | jq .values | jq .[].id); do
    echo $k
done

This prints each individual id as expected.

However, what I want is to access both the email and the id in the loop.

I tried to assign values to SHELL variables like below -

emails=$(cat input.json | jq .values | jq .[].email)
ids=$(cat input.json | jq .values | jq .[].id)

This could work for the most part but ids can have spaces too which is breaking this.

I could essentially have to 2 for loops one for email and the other for id and assign values to arrays in the loop

   i=0
    for k in $(cat input.json | jq .values | jq .[].id); do
            ids[$i]=$k
            i=$(($i +1)) 
        done

and

i=0
for k in $(cat input.json | jq .values | jq .[].email); do
        emails[$i]=$k
        i=$(($i +1)) 
    done

Once I have both the values in arrays, I could parallely traverse both of them.

I am not a shell expert so I wanted to know if there is any slick way of doing this with fewer loops/ lines of code.

Thanks for any help!

BTW, cat foo | anything is pretty much always better replaced with anything <foo or, when the program supports it, anything foo. Either of these gives anything a direct file handle on foo, so it can seek around, read out-of-order, etc; this lets sort split into different threads to read and sort different parts of the file in parallel, or tail jump straight to the end, etc. By contrast, cat only reads front-to-back in-order, with no ability to skip. — Charles Duffy
– Charles Duffy, Commented May 10, 2019 at 17:37
...and for anything in $(...) is itself inherently buggy; there are some differences between bash and zsh, but much of BashPitfalls #1 applies. — Charles Duffy
– Charles Duffy, Commented May 10, 2019 at 17:39

oguz ismail · Accepted Answer · 2021-01-21 07:24:57Z

4

You can output each email-ID pair as a comma separated list from JQ, and use read them into variables in a while loop like so:

while IFS=',' read -r email id; do
  echo "$email"
  echo "$id"
done <<EOF
$(jq -r '.values[] | "\(.email),\(.id)"' file)
EOF

edited Jan 21, 2021 at 7:24

answered May 10, 2019 at 16:45

oguz ismail

51.8k16 gold badges64 silver badges83 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

peak · Accepted Answer · 2019-05-11 20:04:37Z

2

Assuming the variables don't have embedded carriage returns, you can save yourself a lot of IFS grief by having separate read commands for each variable, e.g.:

jq -r '.values[] | (.email, .id)'  input.json |
while IFS= read -r email ; do
    IFS= read -r id 
    echo "email=$email and id=$id"
done

edited May 11, 2019 at 20:04

answered May 11, 2019 at 4:00

peak

119k21 gold badges185 silver badges218 bronze badges

Collectives™ on Stack Overflow

Extract values from JSON objects and assign them to shell variables in a loop

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related