Creating a JSON object with name of the JSON file and its content in bash

Question

I have a few .json files in my directory. If I do:

find . -name '*.json' -exec echo "{\"filename\": \"{}\", \"content\": `cat {}` }," \;

I get:

{"filename": "./a.json", "content":  },
{"filename": "./b.json", "content":  },

However, if I do:

find . -name '*.json' -exec echo "{\"filename\": \"{}\", \"content\": cat {} }," \;

I get:

{"filename": "./a.json", "content": cat ./a.json },
{"filename": "./b.json", "content": cat ./b.json },

so how do I make the a.json and b.json contents be cated correctly ?

BTW, if I do:

find . -name '*.json' -exec cat {} \;

the json files are correctly printed in the console, so I know that the file contents are valid.

Don't try to create JSON by concatenating strings. Use the the jq utility. — Barmar
– Barmar, Commented Jun 2, 2020 at 16:27
I will be happy to accept an answer with jq too for what I am trying to do. — Sankar
– Sankar, Commented Jun 2, 2020 at 16:36
@Inian It will be in multiple sub-directories and the filenames will also contain spaces. — Sankar
– Sankar, Commented Jun 2, 2020 at 16:39

Inian · Accepted Answer · 2020-06-03 06:11:17Z

7

Using bash without find you can do below. You can use nullglob and globstar for recursive JSON file name traversal

shopt -s nullglob globstar

By default jq provides a function input_filename which prints the filename being processed currently. The documentation says, it should work fine as long as the locale is UTF-8

jq '{ filename: input_filename, content: . }' **/*.json

To have a comma between the object entries, use the -n flag with inputs and put the entire result in an array [..]

jq -n '[inputs | { filename: input_filename, content: . }]' **/*.json

For compact output on the same line, use the -c flag as jq -c '..'

edited Jun 3, 2020 at 6:11

answered Jun 2, 2020 at 16:44

Inian

87.1k15 gold badges167 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Philippe Over a year ago

-n use 'null' as the single input value; could you explain briefly how -n flag works in the context of the last command ?

Inian Over a year ago

@Philippe: The -n construct is often used to construct JSON object from scratch.In the example above filename and content are created newly. When constructing JSON newly, inputs provides the contents of all the files cat'ed together. Since we are not using slurp mode, the filter is applied on each object and the input_filename gets the name of the file currently processed (internally)

Gilles Quénot · Accepted Answer · 2020-06-02 16:55:29Z

0

With simple bash :

$ cat a.json
{ "a":123 }
$ cat b.json
{ "b":234 }
$ cat "foo/bar/base/c c.json"
{ "c": 456 }
$ shopt -s globstar # enable recursion with '**'
$ for i in **/*.json; do
    cat<<EOF
    { "filename": "$i", "content": $(cat "$i") }
EOF
done | tee file.json
$ cat file.json
{ "filename": "a.json", "content": {"a":123} }
{ "filename": "b.json", "content": {"b":234} }
{ "filename": "foo/bar/base/c c.json", "content": { "c": 456 } }

edited Jun 2, 2020 at 16:55

answered Jun 2, 2020 at 16:32

Gilles Quénot

188k43 gold badges232 silver badges229 bronze badges

9 Comments

Sankar Over a year ago

This does not work if the filenames contain spaces. This is what I first started with. Only when that failed, I went to the find based approach. Also, I have files in sub-directories too.

MarcoLucidi Over a year ago

in this way you "lose" find. I think it would be better if the for iterates over find's output lines

Gilles Quénot Over a year ago

Added file with space in filename, no worries with proper quoting

Gilles Quénot Over a year ago

And no need 'find' here using bash recursion

Sankar Over a year ago

Thanks. Accepted the answer. Just for completion, Can you please add a , in the end of each row ?

|

Collectives™ on Stack Overflow

Creating a JSON object with name of the JSON file and its content in bash

2 Answers 2

2 Comments

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related