Trouble while executing shell command inside shell script

Question

I have a below mentioned Json file. And i need to parse this file with an input argument that i will pass to my shell script. So Basically i need to parse the values which are inside "collection" from the below JSON file. So if i will pass Janan as a input argument then i need Janan_2 as output or if i pass Janan_ex1 then i need Janan_loc_data as output

Input: abc.txt

{"znode":{
    "path":"/aliases.json","prop":{
      "ver":23,
      "aver":10,
      "count":0,
      "time":"Sat Mar 07 04:39:53 GMT 2015 (1425703193598)",
      "cver":0,
      "czxid":123,
      "Owner":0,
      "mtime":"Tue Sep 22 05:59:16 GMT 2015 (1442901556990)",
      "mzxid":123,
      "pzxid":463856470714,
      "dataLength":272},
    "data":"{\"collection\":{\n    \"Janan\":\"Janan_2\",\n    \"Janan_ex1\":\"Janan_loc_data\",\n    \"Neha\":\"Neha_1\",\n    \"cric\":\"cric_2\",\n    \"San\":\"San_1\",\n    \"Arp\":\"Arp_1\",\n    \"Nipun_test4\":\"Nipun_test3\",\n    \"tran_Nipun\":\"tran_Nipun_2\",\n    \"Zing\":\"Zing_1\"}}"},"tree":[{"data":{
        "title":"/aliases.json","attr":{
          "href":"zookeeper?detail=true&path=%2Faliases.json"}}}]}

So actually issue is that i am not able to fetch the values when i am using my command inside my shell and the same command is working fine while running in command line.

Below command is working fine while running at command line

cat abc.txt | awk -v RS=',?\\\\n[[:space:]]+' 'gsub(/\\"/,"")' | cut -d"{" -f3,3 |  cut -d"}" -f1,1 | grep Janan_ex1 | cut -d":" -f2,2

output from above is : Janan_loc_data

But when i am using it inside my shell script, with the below arguments

INPUT=Janan_ex1
alias_1="user/arpan/abc.txt"

alias_2=`cat $alias_1 | awk -v RS=',?\\\\n[[:space:]]+' 'gsub(/\\"/,"")' | cut -d"{" -f3,3 |  cut -d"}" -f1,1 | grep $INPUT | cut -d":" -f2,2`

OUTPUT:-

\atm_2\,\n    \atm_ex1\

This is because it is not able to read special character \ inside shell scripts

For below command when using inside the script,

alias_2=`cat $alias_1 | awk -v RS=',?\\\\n[[:space:]]+' 'gsub(/\\"/,"")' | cut -d"{" -f3,3 |  cut -d"}" -f1,1 | grep $INPUT`

I am getting below output:-

 \n    \Janan\:\Janan_2\,\n    \Janan_ex1\:\Janan_loc_data\,\n    \dps\:\dps_1\,\n    \ammf\:\ammf_2\,\n    \mob_search\:\mob_search_1\,\n    \dnb\:\dnb_1\,\n    \stamping_test4\:\stamping_test3\,\n    \tran_stamping\:\tran_stamping_2\,\n    \mrch_generic\:\mrch_generic_1\

Also i cannot use json parser inside my script as it is not installed in machine

Details regarding my unix flavor

$ uname -a
Linux 2.6.32-573.3.1.el6.x86_64 #1 SMP Mon Aug 10 09:44:54 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

Then install a JSON parser (like jq). awk was designed for line-oriented text, not structured input like JSON. — chepner
– chepner, Commented Sep 24, 2015 at 18:33
Chepner It is not possible to install at our production env..needs lots of approval and take lots of days to do it and i need it by EOD today. — Arpan
– Arpan, Commented Sep 24, 2015 at 18:42
Without a structured tool try an unstructured approach. Something like grep collection abc.txt | sed -e 's/[[:space:]]\\"/\n/g' -e 's/}}"},.*//' -e 's/\\",\\n//g' -e 's/\\"//g' and see if you can use the output for the translation. — Walter A
– Walter A, Commented Sep 24, 2015 at 21:08

chepner · Accepted Answer · 2015-09-24 18:53:09Z

1

I strongly suggest you install a JSON parser; it's the right tool for the job. Here's how you would solve your problem with jq:

$ jq '.znode.data | fromjson | .collection.Janon' abc.txt
"Janan_2"
$ jq '.znode.data | fromjson | .collection.Janon_ex1' abc.txt
"Janan_loc_data"

In general, you could use the --arg option:

$ jq --arg field Janan '.znode.data | fromjson | .collection | .[$field]' abc.txt
"Janan_2"

For now, try using $(...) instead of backticks; they treat backslashes slightly differently. Using your code, I get an output of \Janan_loc_data\, but with the following, I get Janan_loc_data.

alias_2=$(awk -v RS=',?\\\\n[[:space:]]+' 'gsub(/\\"/,"")' "$alias_1" |
           cut -d"{" -f3,3 |  cut -d"}" -f1,1 | grep "$INPUT" | cut -d":" -f2,2)

edited Sep 24, 2015 at 18:53

answered Sep 24, 2015 at 18:40

chepner

538k77 gold badges594 silver badges746 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Arpan Over a year ago

Hi Chepner, i have already mentioned this to our unix admin ..but not getting a proper response ..as of now i cannot use it :(

user000001 Over a year ago

@Arpan If you compile it from source you can probably install it in your home directory without sudo

Arpan Over a year ago

@user000001 : But it will only hold good for my dev environment ..it wont work for production as jq is not installed at production as well. .. Plus there are lots of restriction here while doing any new installation ..

Collectives™ on Stack Overflow

Trouble while executing shell command inside shell script

Also i cannot use json parser inside my script as it is not installed in machine

Details regarding my unix flavor

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Also i cannot use json parser inside my script as it is not installed in machine

Details regarding my unix flavor

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related