Get alternative elements in JSON using JQ (output: JSON object)

Question

I have a problem where I'm receiving JSON that depending on the source, may have some data at root level or under another element.

From this JSON I need to extract multiple elements into a nother, normalized JSON, with custom keys for the values.

Sample JSON 1:

{"name":"John","last":"Smith","position": "clerk"}

Sample JSON 2:

{"personData": {"name":"John","last":"Smith"}}

I was using this at the beginning as all my test jsons had the first form:

jq '{nombre: .name, apellido: .last, puesto: .position}'

When they started coming up empty I tries using the // operator but it doesn't seem to work if I'm providing the keys:

Doesn't work:

jq '{nombre: .name//.personData.name, apellido: .last, puesto: .position}'

Works (but I don't get any keys):

jq '{nombre: .name//.personData.name, apellido: .last, puesto: .position//null}'

I've never had to use jq with conditionals or logic like this, so I may be missing something obvious. (actual json files are pretty extensive and whichever logic I need to use will end up applied to a dozen or so fields, so I'm looking for a sustainable solution.

Expected output would be for the first sample:

{"nombre" : "John", "apellido": "Smith", "puesto": "clerk" }

And for the second sample:

{"nombre" : "John", "apellido": "Smith", "puesto": null}

Answer: Ended up using this style, which allows for searching in multiple blocks for extracting multiple data:

(.personData // .) + (.addressData // .) | {nameValue: .name, streetValue: .streetName }

(simplified: The .name value can be at root or under .personData.name, and the .streetName value can be at root or under .addressData.streetvalue, so the above normalizes the situation for both blocks into the same stream)

peak · Accepted Answer · 2023-05-07 19:40:34Z

3

One of many possibilities:

(.personData // .) |  {nombre: .name, apellido: .last, puesto: .position}

If you want a fully exhaustive search for all JSON objects with .name, then you could use .. or walk/1. If you want at most one such object, you could use first/1, which might be a good idea anyway as it might speed things up.

edited May 7, 2023 at 19:40

answered May 7, 2023 at 1:17

peak

119k21 gold badges185 silver badges218 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Eduo Over a year ago

Ok, if I understand this what you’re doing here is setting the filter to be for the child element and if it’s not present then use the root element, right?

peak Over a year ago

Yes. See the man page or for more details the FAQ at github.com/stedolan/jq/wiki/FAQ#or-versus-

Eduo Over a year ago

Used a version of this, since multiple data was in various places: (.personData // .) + (.addressData // .) | {nameValue: .name, streetValue: .streetName } Thanks!

peak Over a year ago

(.personData // .addressData // ..) would be briefer and probably better.

Gilles Quénot · Accepted Answer · 2023-05-07 17:10:07Z

1

One way, using recursion ..:

.. | select(has("name") and has("last"))? |
    { nombre: .name, appelido: .last, puesto: .position }

Can be re-used in other contexts.

From man jq:

Recursive Descent: ..
Recursively descends ., producing every value. This is the same as the zero-argument recurse builtin. This is intended to resemble the XPath // operator.

edited May 7, 2023 at 17:10

answered May 7, 2023 at 1:32

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

5 Comments

Eduo Over a year ago

Thanks! Is this checking for the key at all children (..)? Interesting, as it avoids the condition and covers possible additional scenarios of depth. What is the [-1] at the end for? It would normally be for getting the last element of an array, no?

Gilles Quénot Over a year ago

Yes, last element of array of objects

Eduo Over a year ago

Ah. I understand. This is multiplying the results each time, which is why you're getting the last element. This is useful, but it won't work because the number of fields increases a lot and the processing time gets doubled each time. I added several more fields and did a wc -l and it was already 325 thousand lines. It becomes tremendously slow. Thanks anyway! I didn't know about this operator and it will be very useful for other things!

Gilles Quénot Over a year ago

From man jq it's inspired from XPath : // recursive operator

Gilles Quénot Over a year ago

Post edited to do it properly in a recursive way.

Collectives™ on Stack Overflow

Get alternative elements in JSON using JQ (output: JSON object)

2 Answers 2

4 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related