How can I use keys and values from jq to generate a bash associative array?

Question

From Json structure, I want a specific dictionary. From key's color yellow or red, I add id value.

[
  {
    "id": "9b058640",
    "type": "db",
    "color": "red",
    "host": "db1"
  },
  {
    "id": "0u858640",
    "type": "db",
    "color": "yellow",
    "host": "db2"
  },
  {
    "id": "0ui9k40",
    "type": "net",
    "color": "red",
    "host": "net1"
  },
  {
    "id": "5ty87a",
    "type": "net",
    "color": "yellow",
    "host": "net2"
  }
]

So I want to get the X dictionary

X=(
   ['yellow']="9b058640 5ty87a"
   ['red']="9b058640 0ui9k40"
 )

I could parse by value :

jq -c '.[] | select(.color | contains("red"))'

pmf · Accepted Answer · 2022-02-21 12:31:08Z

4

No need for ., source, eval or even looping in bash. All you need is declare and jq, which can construct the declaration using escaping with @sh and string interpolation:

declare -A X="($(
  jq -r '
    ("yellow", "red") as $color 
    | @sh "[\($color)]=\(map(select(.color == $color).id) | join(" "))"
  ' input.json
))"

$ echo "${X[yellow]}"
0u858640 5ty87a

$ echo "${X[red]}"
9b058640 0ui9k40

edited Feb 21, 2022 at 12:31

answered Feb 20, 2022 at 20:37

pmf

38.5k3 gold badges31 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

marcooo_i Over a year ago

Thanks you very, it works, it was impossible to do it by myself !!

bobmarksie Over a year ago

Excellent answer! jq really is an amazing tool, so well designed / considered - so impressed with it and what can be done with simple bash / jq

Charles Duffy · Accepted Answer · 2022-02-20 18:22:24Z

1

You can use the @tsv operator in jq to emit tab-separated output, which a bash while read loop can easily parse as input.

Assuming your input JSON is in the variable s:

declare -A X=( )

while IFS=$'\t' read -r color id; do
  X[$color]+="$id "
done < <(jq -r '.[] | [.color, .id] | @tsv' <<<"$s")

The above does take a minor shortcut in that it leaves a trailing space after each item. If that's unacceptable for some reason, you can always go through the array and clean it up after the fact with a second loop:

for color in "${!X[@]}"; do
  X[$color]=${X[$color]%" "}
done

You can see this running in the sandbox at https://replit.com/@CharlesDuffy2/IndigoRemoteEngineering

Alternately, using eval:

#!/usr/bin/env bash
case $BASH_VERSION in ''|[0-3].*) echo "ERROR: bash 4.0+ required" >&2; exit 1;; esac
declare -A X=( )
eval "$(
  jq -r '
    reduce .[] as $item ({}; .[$item.color] += [$item.id])
    | to_entries[]
    | "X[\(.key | @sh)]=\(.value | join(" ") | @sh)"
  '
)" <file.json

edited Feb 20, 2022 at 18:22

answered Feb 20, 2022 at 17:36

Charles Duffy

299k43 gold badges442 silver badges498 bronze badges

2 Comments

marcooo_i Over a year ago

Thanks very much for you replying, but I edit the question because of structure of json, because there is comma , and bracket []

Charles Duffy Over a year ago

In that case just .[] | needs to be added to the query; updating appropriately.

dan · Accepted Answer · 2022-02-20 19:08:36Z

0

Answer:

declare -A a
. <(jq -r '"a[\(.color)]+=${a[\(.color)]+ }\(.id)"' file.json)

Provides:

$ declare -p a
$ declare -A a=([red]="9b058640 0ui9k40" [yellow]="0u858640 5ty87a" )
# or
$ echo "${a[red]}"; echo "${a[yellow]}"
9b058640 0ui9k40
0u858640 5ty87a

Explanation:

Construct the following bash code in jq:
a[color]+=${a[color]+ }id
${a[color]+ } expands to , only if a[color] is empty.
"\(.id)" is jq string interpolation - its replaced with the value of .id
use . and a process sub to source this code

edited Feb 20, 2022 at 19:08

answered Feb 20, 2022 at 17:39

dan

5,4319 silver badges17 bronze badges

6 Comments

Charles Duffy Over a year ago

Using eval, source or . on generated code introduces security exposure that needs to be mitigated: Use | @sh inside the \( ... ) segments to ensure that hostile IDs or color names can't be used to attack the system's security. You don't want a JSON file saying that something is colored $(rm -rf ~) to delete your home directory.

marcooo_i Over a year ago

Thanks very much for you replying, but I edit the question because of structure of json, because there is comma , and bracket []

dan Over a year ago

@CharlesDuffy The issues is escaping .color inside the array subscript. The quotes aren't removed. However a I found a version that works similarly.

Charles Duffy Over a year ago

@dan, ...I added my own eval-based version that avoids the whitespace issue -- see addendum to other answer.

Charles Duffy Over a year ago

@dan, see replit.com/@CharlesDuffy2/SphericalWhirlwindDebuggers#main.sh -- yes, there's an error message, but the injected code still gets run. And if you made it $(rm -rf ~; echo hi), then there wouldn't be an error message at all -- the reason there was an error message is that the stdout from rm was empty, and an empty string isn't a valid key.

|

Collectives™ on Stack Overflow

How can I use keys and values from jq to generate a bash associative array?

3 Answers 3

2 Comments

2 Comments

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related