Parsing a curl json response and using response to build another request

Question

I am trying to build an shell script that will fetch the image url for a random XKCD comic, so that I can display it using Übersicht. Since you can only ask the XKCD API for the latest comic or a specific comic I need to:

Send a GET request to http://xkcd.com/info.0.json, fetch the value of the num element.
Send another request to http://xkcd.com/XXX/info.0.json where XXXis the value of num.

My current command looks like this and successfully returns the comic number:

curl -s 'http://xkcd.com/1510/info.0.json' | grep -Eo '"num": \d+' | grep -Eo '\d+'

I haven't been able to figure out how to use capture groups with grep, so I need to grep the json twice. The common advice is to use -P, which is not supported in Mac OS X 10.10.
I have no idea how to read the output of grep (as XXX) into the second curl -s 'http://xkcd.com/XXX/info.0.json' command.

4ae1e1 · Accepted Answer · 2017-03-01 18:46:39Z

11

On OS X you can use the system Perl:

 curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/'

You can save the output to a variable with command substitution:

num=$(curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/')
curl -sS "http://xkcd.com/${num}/info.0.json"

or more concisely, two-in-one, albeit not very readable:

curl -sS "http://xkcd.com/$(curl -sS http://xkcd.com/info.0.json | /usr/bin/perl -pe 's/.*"num": ([0-9]+).*/\1/')/info.0.json"

By the way, I highly recommend jq as the command line JSON processor. To extract num with jq, it's as simple as

curl -sS http://xkcd.com/info.0.json | jq '.num'

and although you didn't ask for it, here's a simple one-liner with jq that extracts the URI of the latest image:

curl -sS "http://xkcd.com/$(curl -sS http://xkcd.com/info.0.json | jq '.num')/info.0.json" | jq -r '.img'

Example output:

http://imgs.xkcd.com/comics/spice_girl.png

edited Mar 1, 2017 at 18:46

answered Apr 14, 2015 at 7:51

4ae1e1

7,7229 gold badges51 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Samuel Lindblom Over a year ago

This works great in the Terminal, so I'm accepting it. Thanks, great work. Now I just need to get it working in Übersicht - I am getting an octal escape sequence error.

4ae1e1 Over a year ago

@SamuelLindblom I can't see any octal escape sequence in my commands though. And I don't use Übersicht — can you explain which step goes wrong in detail? (If you're not sure, you can break down the pipeline and test separately.)

Collectives™ on Stack Overflow

Parsing a curl json response and using response to build another request

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related