Loop through json variable

Question

I have a json variable which looks like this:

{
  "status": "closed",
  "host": "host1-availabe_zone_A"
}
{
  "status": "closed",
  "host": "host2-availabe_zone_B"
}
{
  "status": "closed",
  "host": "host3-availabe_zone_A"
}

I am trying to convert it to a hash-like data structure (associate array in zsh/bash) to get number of hosts in each availabe zone. The preferred result will be this based on the data above:

availabe_zone_A -> 2
availabe_zone_B -> 1

I got some problem when trying to iterate the json variable.

declare -A az_to_number_of_open_hosts
for host in $(jq . <<<  $hosts_list)
do
  az=$(rev <<< $(echo $host | jq .host) | cut -d. -f3 | rev)
  ((az_to_number_of_open_hosts[$az]++))
done

The line to extract available zone is because the host format is always blabla123213.az.domain.com. The code does not work since it seems the variable host refers to each line of json string rather than a json entry. Is there a recommended way to iterate each json entry?

That's not valid JSON. If you have multiple objects they have to be contained in an array so you can loop through it. — Barmar
– Barmar, Commented Aug 28, 2019 at 15:53
What are you expecting jq . to do? It just returns the input as its output. — Barmar
– Barmar, Commented Aug 28, 2019 at 15:56
Why do you declare az_to_number_of_open_hosts and not use it? — Barmar
– Barmar, Commented Aug 28, 2019 at 15:57
@Barmar Isn't it a valid json? since jq works pretty well with the entries above: jq .host /tmp/test.json "host1-availabe_zone_A" "host2-availabe_zone_B" "host3-availabe_zone_A" — dashenswen
– dashenswen, Commented Aug 28, 2019 at 16:00
jq can iterate over a stream of separate JSON values, which is what you have. — chepner
– chepner, Commented Aug 28, 2019 at 16:49

Jeff Mercado · Accepted Answer · 2019-08-28 17:32:25Z

1

You're just counting values in your json. Split out the part you want to count

$ jq -nr 'inputs.host/"-"|last' input.json
availabe_zone_A
availabe_zone_B
availabe_zone_A

then you could either group and count:

$ jq -nr '[inputs.host/"-"|last]|group_by(.)[]|"\(.[0]) -> \(length)"' input.json
availabe_zone_A -> 2
availabe_zone_B -> 1

https://jqplay.org/s/g_8HgqiQ1o

or iterate the values tallying as you go:

$ jq -nr '
reduce (inputs.host/"-"|last) as $k ({}; .[$k] += 1)|to_entries[]|"\(.key) -> \(.value)"
' input.json
availabe_zone_A -> 2
availabe_zone_B -> 1

https://jqplay.org/s/S2oeLfMRNR

answered Aug 28, 2019 at 17:32

Jeff Mercado

136k34 gold badges268 silver badges282 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Barmar · Accepted Answer · 2019-08-28 16:05:48Z

0

Use .host to get the host properties.

$ hosts_list='  {
    "status": "closed",
    "host": "host1-availabe_zone_A"
  }
  {
    "status": "closed",
    "host": "host2-availabe_zone_B"
  }
  {
    "status": "closed",
    "host": "host3-availabe_zone_A"
  }'
$ for host in $(jq -r '.host' <<<  "$hosts_list")
do
 echo "$host"
done
host1-availabe_zone_A
host2-availabe_zone_B
host3-availabe_zone_A

Use the -r option to make it output raw strings, rather than JSON strings surrounded by double quotes.

And don't forget to quote variables unless you deliberately need word-splitting and wildcard matching to be done on the value.

answered Aug 28, 2019 at 16:05

Barmar

789k57 gold badges555 silver badges669 bronze badges

3 Comments

peak Over a year ago

This does not provide a solution to the stated problem (as it exists right now). Also, if the user just wished to print the key names, there would still be no need to use a shell loop. –

Barmar Over a year ago

@peak he edited the question after I wrote the answer, originally it just printed them. That was presumably just a simplification of what he really wanted to do with the names.

peak Over a year ago

Yes, that is why I included the parenthetical remark.

peak · Accepted Answer · 2019-08-28 17:48:08Z

0

With the following in program.jq:

# bag of words
def bow(stream): 
  reduce stream as $word ({}; .[($word|tostring)] += 1);

bow(inputs | .host | sub("^.*-availabe";"availabe"))

the invocation:

jq -n -f program.jq input.json

produces

{
  "availabe_zone_A": 2,
  "availabe_zone_B": 1
}

If you really want the output in the STRING --> N format, add -r to the command-line options, and add the following two lines to program.jq:

| to_entries[]
| "\(.key) --> \(.value)"

Comments

You might want to fix the spelling ("available" vs "availabe")
As this example illustrates, jq can handle an input stream consisting of JSON entities -- i.e. a stream of JSON.

edited Aug 28, 2019 at 17:48

answered Aug 28, 2019 at 17:39

peak

119k21 gold badges185 silver badges218 bronze badges

Collectives™ on Stack Overflow

Loop through json variable

3 Answers 3

Comments

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related