63

I have this script:

nmapout=`sudo nmap -sP 10.0.0.0/24`
names=`echo "$nmapout" | grep "MAC" | grep -o '(.\+)'`
echo "$names"

now the $names variable contains strings delimited with newlines:

>_
 (Netgear)
 (Hon Hai Precision Ind. Co.)
 (Apple)

I tried to do the array conversion with the sub-string approach:

names=(${names//\\n/ })
echo "${names[@]}"

But the problem is that I can't access them by indexing (i.e., ${names[$i] etc.), if I run this loop

for (( i=0; i<${#names[@]}; i++ ))
do
     echo "$i: ${names[$i]"
     # do some processing with ${names[$i]}
done

I get this output:

>_
 0: (Netgear)
 1: (Hon
 2: Hai

but what I want is:

>_
 0: (Netgear)
 1: (Hon Hai Precision Ind. Co.)
 2: (Apple)

I could not figure out a good way to do this, please note that the second string has spaces in it.

3
  • Any reason why do you want array? I would prefer to use read by line loop. Commented Jul 8, 2014 at 9:31
  • @kan , actually this is a small portion of a large script, the original script uses the index for other purposes, that's why I want to keep the array. Commented Jul 8, 2014 at 16:09
  • Related: how to convert a space-delimited string to a bash array: Reading a delimited string into an array in Bash Commented Dec 17, 2021 at 20:53

6 Answers 6

100

Set IFS (Internal Field Separator). Shell uses the IFS variable to determine what the field separators are. By default, IFS is set to the space character. Change it to the newline character, as demonstrated below:

#!/bin/bash
names="Netgear
Hon Hai Precision Ind. Co.
Apple"
    
SAVEIFS=$IFS   # Save current IFS (Internal Field Separator)
IFS=$'\n'      # Change IFS to newline char
names=($names) # split the `names` string into an array by the same name
IFS=$SAVEIFS   # Restore original IFS

for (( i=0; i<${#names[@]}; i++ ))
do
    echo "$i: ${names[$i]}"
done

Output

0: Netgear
1: Hon Hai Precision Ind. Co.
2: Apple
Sign up to request clarification or add additional context in comments.

13 Comments

Because there could be special characters in the original $IFS, it's better to avoid trying to store it. Better to just wrap the whole thing in a subshell with parentheses.
@etheranger , I am new in bash scripting, could you please elaborate more on "subshell with parantheses"?
If i execute that as non-root, I get Syntax error: "(" unexpected
you can change IFS just for one line using IFS=$'\n' names=(${names}) on line 9. It's the same as joining line 8 and line 9.
@GabrielStaples man bash section QUOTING: "Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard."
|
47

Bash also has a readarray builtin command, easily searchable in the man page. It uses newline (\n) as the default delimiter, and MAPFILE as the default array, so one can do just like so:

    names="Netgear
    Hon Hai Precision Ind. Co.
    Apple"

    readarray -t <<<$names

    printf "0: ${MAPFILE[0]}\n1: ${MAPFILE[1]}\n2: ${MAPFILE[2]}\n"

The -t option removes the delimiter ('\n'), so that it can be explicitly added in printf. The output is:

    0: Netgear
    1: Hon Hai Precision Ind. Co.
    2: Apple

3 Comments

This is the correct answer to the question that was asked. readarray is designed to do exactly this
This indeed is the correct answer to the specific question.
readarray was introduced in bash v4.0. Some systems like macOS <11.* are still on bash v3.2. In that case IFS-based solutions can used instead.
26

Let me contribute to Sanket Parmar's answer. If you can extract string splitting and processing into a separate function, there is no need to save and restore $IFS — use local instead:

#!/bin/bash

function print_with_line_numbers {
    local IFS=$'\n'
    local lines=($1)
    local i
    for (( i=0; i<${#lines[@]}; i++ )) ; do
        echo "$i: ${lines[$i]}"
    done
}

names="Netgear
Hon Hai Precision Ind. Co.
Apple"

print_with_line_numbers "$names"

See also:

Comments

8

How to read a multi-line string into a regular bash "indexed" array

The Bash shellcheck static code analyzer and checker tool recommends in SC2206 to use read -r or mapfile. Their mapfile example is complete, but their read example only covers the case of splitting a string by spaces, not newlines, so I learned the complete form of the read command for this purpose from @Toni Dietze's comment here.

So, here is how to use both to split a string by newlines. Note that <<< is called a "herestring". It is similar to << which is a "heredoc", and < which reads in a file:

# split the multiline string stored in variable `var` by newlines, and
# store it into array `myarray`

# Option 1
# - this technique will KEEP empty lines as elements in the array!
# ie: you may end up with some elements being **empty strings**!
mapfile -t myarray <<< "$multiline_string"

# OR: Option 2 [my preference]
# - this technique will NOT keep empty lines as elements in the array!
# ie: you will NOT end up with any elements which are empty strings!
IFS=$'\n' read -r -d '' -a myarray <<< "$multiline_string"

There is also a 3rd technique I use the most, which is not necessarily recommended by shellcheck, but which is fine if you use it correctly, and which is far more readable than either of the options above. I use it in many scripts in my eRCaGuy_dotfiles/useful_scripts directory here. Clone that repo and run grep -rn "IFS" in it to find all places where I use that technique.

See here for where I first learned this: Answer here by @Sanket Parmar: Convert multiline string to array.

Here it is:

# Option 3 [not necessarily recommended by shellcheck perhaps, since you must
# NOT use quotes around the right-hand variable, but it is **much
# easier to read**, and one I very commonly use!]
#
# Convert any multi-line string to an "indexed array" of elements:
#
# See:
# 1. "eRCaGuy_dotfiles/useful_scripts/find_and_replace.sh" for an example 
#    of this.
# 1. *****where I first learned it: https://stackoverflow.com/a/24628676/4561887
SAVEIFS=$IFS   # Save current IFS (Internal Field Separator).
IFS=$'\n'      # Change IFS (Internal Field Separator) to the newline char.
# Split a long string into a bash "indexed array" (via the parenthesis),
# separating by IFS (newline chars); notice that you must intentionally NOT use
# quotes around the parenthesis and variable here for this to work!
myarray=($multiline_string) 
IFS=$SAVEIFS   # Restore IFS

See also:

  1. Where I learned my "Option 3" above: Answer here by @Sanket Parmar: Convert multiline string to array
  2. Read file into array with empty lines
  3. An example where I read a bash multi-line string into a bash array using the read cmd: Find all files in a directory that are not directories themselves

Comments

7

As others said, IFS will help you.IFS=$'\n' read -ra array <<< "$names" if your variable has string with spaces, put it between double quotes. Now you can easily take all values in a array by ${array[@]}

5 Comments

By default, read uses \n as delimiter, so you have to put -d '' in the read command, otherwise the array only contains the first line of $names. Corrected version: IFS=$'\n' read -r -d '' -a array <<< "$names". You also forgot to put a $ in front the {.
I am new to this, Could you elaborate more about -r and -a usage in this command
I am a bit confused. You already use -r and -a in your initial answer, just shortened to -ra. In my comment, I added -d ''. The bash man page nicely explains all these command line options (look for the read builtin command).
@ToniDietze, thanks for your corrections! I never would have figured out to add -d '' otherwise, and that part is essential. I added it to my answer here.
It's worth mentioning that the read builtin will return a non-zero exit status upon encountering an EOF, so if you have set -e somewhere in your shell script as the Bash Strict Mode document suggests, you better mask the exit code of read, e. g.: read -ra array -d '' <<< "${names}" || true.
2

Adding the needed null byte delimiter in @HariBharathi answer

#!/bin/bash

IFS=$'\n' read -r -d '' -a array <<< "$names"

Remark: Unlike mapfile/readarray, this one is compatible with macOS bash 3.2

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.