Expanding bash variable within a string

Question

I am trying to get a variable within a string to expand - its a path:

/path/to/${variable}/here

The string originates in a json file.
It is subsequently read by jq - at this point I need the expansion to occur.

Here is the json file:

{
    "server_1":{
      "tag": "storage_1",
      "user": "me",
      "sshKey": "/Users/me/.ssh/id_rsa",
      "target_folder": "/Users/me/Desktop/",
      "pubIp": "127.0.0.1",
      "data_folders": [
        "/Users/me/Desktop/POD_INSTALLS/pod_DB/${BUILD_FOLDER}/data/table1/",
        "/Users/me/Desktop/POD_INSTALLS/pod_DB/${BUILD_FOLDER}/data/table2/"
      ]
    }
}

Here is the jq code (now with working code - thanks all):

  writeFolder=$(jq -r --arg bf "${BUILD_FOLDER}" '.server_'${id}'.data_folders[] | sub("\\${BUILD_FOLDER}";$bf)' "${json_path}")
  for folder in $writeFolder
  do
    ssh -q -o ForwardX11=no -i ${sshKey} ${user}@${pubIp} "mkdir -p ${folder}"
  done

Originally - within the expanded $writeFolder variable, the path could be seen but the ${BUILD_FOLDER} variable remained unexpanded as it was taken literally. (when I used set -x I could see it was surrounded by single quotes).

Eventually used accepted solution from peak to make better use of jq.

In the example shown there are only two paths inside data_folders, so are you planning to do 2-3 and get -1 for the value of numberOfDataFolders? — Inian
– Inian, Commented Feb 21, 2018 at 17:43
In the JSON you show, data_folders is a sibling, not a member, of server_1. — chepner
– chepner, Commented Feb 21, 2018 at 17:49
You can also push more work into jq, avoiding shell interpolation and pipelines. For example, numberOfDataFolders=$( jq --arg id "$id" '.["server_" + $id].data_folders | length - 3'). — chepner
– chepner, Commented Feb 21, 2018 at 17:51
I think I have seen a similar question, but I cannot find it anymore. The easy but dangerous solution would be something like eval expandedString="$stringWithVariableInside". — Socowi
– Socowi, Commented Feb 21, 2018 at 18:22
thanks for all your comments - I have corrected my original dodgy copy and paste and have added a working solution suggested by peak. — JRD
– JRD, Commented Feb 22, 2018 at 18:00

peak · Accepted Answer · 2018-02-21 18:51:49Z

1

As pointed out in the comments, you'd be much better off making jq do as much work as possible. But to use jq, you'll need to fix the JSON. Also, it's not clear to me that you need to compute $numberOfDataFolders at all, so in the interests of simplicity, this is what I'd suggest:

#!/bin/bash

json_path=input.json

BUILD_FOLDER=mybuildfolder

jq -r --arg bf "$BUILD_FOLDER" '.data_folders[] | sub("\\${BUILD_FOLDER}";$bf)' "${json_path}" |
  while read -r writeFolder
    do
    echo mkdir -p "${writeFolder}"
  done

Once the input has been fixed, this produces:

mkdir -p /Users/me/Desktop/POD_INSTALLS/pod_DB/mybuildfolder/data/table1/
mkdir -p /Users/me/Desktop/POD_INSTALLS/pod_DB/mybuildfolder/data/table2/

Notice also that the -r option circumvents the need to use tr.

edited Feb 21, 2018 at 18:51

answered Feb 21, 2018 at 18:45

peak

119k21 gold badges185 silver badges218 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Expanding bash variable within a string

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related