Bash - variables are not evaluated inside other variables

Question

I've written a short script which needs to find some text using regex.

I'm incrementing a counter inside a while loop, and this counter is part of another command. Unfortunately this command is always running with the initial counter.

Here a snippet from my code:

COUNTER=1
LAST_COMMIT=`git log remotes/origin/devel --pretty=oneline --pretty=format:%s | head -${COUNTER}`
JIRA_ID=`echo $LAST_COMMIT | grep -o -P '[A-Z]{2,}-\d+' | xargs`

while [[ ! -z "$JIRA_ID" && $COUNTER -lt "5" ]]; do
        echo "This is the current  counter:  $COUNTER"
        echo "This is the last commit $LAST_COMMIT"
        COUNTER=$[COUNTER+1]
done
echo "this is the counter outside the loop $COUNTER"

You only run the command once. With the initial value of $COUNTER. If you need to run it with a different value then you need to run it again. Neither LAST_COMMIT nor JIRA_ID ever change once set. Also | xargs is likely pointless there as it is just /bin/echo. — Etan Reisner
– Etan Reisner, Commented Jul 23, 2015 at 16:00
This might help you; unix.stackexchange.com/questions/60688/… — AlG
– AlG, Commented Jul 23, 2015 at 16:08
Every time you need to get a new LAST_COMMIT value you need to run that command. Not just once before the loop. Same thing for every time you want to get a new JIRA_ID value (from a new LAST_COMMIT value). — Etan Reisner
– Etan Reisner, Commented Jul 23, 2015 at 16:08
Also you can tell git log how many entries to output directly you don't need head for that. And this is going to get you increasingly many lines of output from git log each time your COUNTER increments and I don't think that's what you want. (Also I think --pretty=format is overriding --pretty=oneline so you probably don't need both.) — Etan Reisner
– Etan Reisner, Commented Jul 23, 2015 at 16:10

Charles Duffy · Accepted Answer · 2015-07-23 20:20:57Z

2

The best-practices way to encapsulate code (as per BashFAQ #50) is with a function:

get_last_commit() {
  git log remotes/origin/devel --pretty=oneline --pretty=format:%s \
    | sed -n "$(( $1 + 1)) p"
}

Then:

while (( counter < 5 )); do
  last_commit=$(get_last_commit "$counter")
  IFS=$'\n' read -r -d '' -a jira_id \
    < <(grep -o -P '[A-Z]{2,}-\d+' <<<"$last_commit") ||:
  [[ $jira_id ]] || break

  echo "This is the current  counter:  $counter"
  echo "This is the last commit $last_commit"
  echo "Found ${#jira_id[@]} jira IDs"
  printf '  %s\n' "${jira_id[@]}"

  (( counter++ ))
done

Other notes:

Use of read -a, here, reads the JIRA IDs into an array; you can then ask for the array's length (with ${#jira_id[@]}), expand a specific entry from the array (with ${jira_id[0]} to get the first ID, [1] for the second, etc); expand them all into an argument list (with "${jira_id[@]}"), etc.
Non-system-defined shell variables should have at least one lower-case character in their names. See the fourth paragraph of the POSIX spec on environment variables at http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html, keeping in mind that environment variables and shell variables share a single namespace. Following this practice prevents you from overwriting system variables by mistake.
$(( ... )) is the POSIX-standard way to enter a math context; (( )), without a leading $, is a bash extension.
While code inside of [[ ]] and (( )) does not require double quotes to prevent glob expansion or string-splitting, double quotes should be used around expansions in (almost) all other cases.
sed '2 p' gets line 2 more efficiently than head -2 | tail -n 1.

However, even that is much less efficient than just calling git log only one single time and iterating over its results.

while IFS= read -r -u 3 last_commit; do
  IFS=$'\n' read -r -d '' -a jira_id \
    < <(grep -o -P '[A-Z]{2,}-\d+' <<<"$last_commit") ||:
  [[ $jira_id ]] || continue
  echo "Found ${#jira_id[@]} jira IDs"
  printf '  %s\n' "${jira_id[@]}"
done 3< <(git log remotes/origin/devel --pretty=oneline --pretty=format:%s)

edited Jul 23, 2015 at 20:20

answered Jul 23, 2015 at 18:41

Charles Duffy

299k43 gold badges442 silver badges498 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

Alex Brodov Over a year ago

The condition in the while should be , unless the Jira ID is empty continue looping, the counter < 5 was just for demonstration, and I didn't want to get an infinite loop. BTW, how do you suggest avoiding the xargs

Charles Duffy Over a year ago

should be ,? You're missing something. Anyhow, see the [[ $jira_id ]] || break, which causes it to stop looping if jira_id is ever empty after being assigned. Putting that condition in the while itself means you never get into the loop to start with, because the variable is empty before the loop's first iteration runs!

Alex Brodov Over a year ago

Yes but I can't use the counter < 5 as the loop termination, because I don't know how many commits I'll need to scan

Charles Duffy Over a year ago

Then use while true.

Alex Brodov Over a year ago

That's what I thought, what about avoiding the xargs

|

that other guy · Accepted Answer · 2015-07-23 18:42:46Z

0

Bash is a procedural language, meaning it contains a series of steps to be carried out in order.

LAST_COMMIT=`...`is a step that sets the variable LAST_COMMIT to the value of the command. You have made this step only execute once, which is why you see the same value over and over.

If you want it to execute again for new values of $COUNTER, you can place the statement inside the loop:

while
    LAST_COMMIT=`git log remotes/origin/devel --pretty=oneline --pretty=format:%s | head -${COUNTER} | tail -n 1`
    JIRA_ID=`echo $LAST_COMMIT | grep -o -P '[A-Z]{2,}-\d+' | xargs`
    [[ ! -z "$JIRA_ID" && $COUNTER -lt "5" ]]
do

    echo "This is the current  counter:  $COUNTER"
    echo "This is the last commit $LAST_COMMIT"
    COUNTER=$[COUNTER+1]
done
echo "this is the counter outside the loop $COUNTER"

Since the steps in the loop are executed multiple times, the LAST_COMMIT=`...` step is also executed multiple times, each time with a new value for $COUNTER.

edited Jul 23, 2015 at 18:42

answered Jul 23, 2015 at 18:37

that other guy

125k12 gold badges187 silver badges214 bronze badges

4 Comments

Alex Brodov Over a year ago

I think that it's a workaround, because the while shouldn't be running if the first commit contains the JIRA ID, correct me if I'm wrong

that other guy Over a year ago

@user3502786 I didn't notice that, but it's still the same concept. You need the statements in the loop. I've updated it.

Alex Brodov Over a year ago

So we'd need to make a duplicate code here , which is not good

that other guy Over a year ago

@user3502786 Where are you seeing the duplicate code?

Collectives™ on Stack Overflow

Bash - variables are not evaluated inside other variables

2 Answers 2

13 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

13 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related