Diff for "BashFAQ/005"

How can I use array variables?

BASH and KornShell have one-dimensional arrays indexed by a numerical expression, e.g.:

•  # Bash
   host[0]="micky"
   host[1]="minnie"
   host[2]="goofy"
   i=0
   while (( $i < ${#host[@]} ))
   do
       echo "host number $i is ${host[i++]}"
   done

The indexing always begins with 0.

The awkward expression ${#host[@]} returns the number of elements for the array host. Also noteworthy for BASH is the fact that inside the square brackets, i++ works as a C programmer would expect. The square brackets in an array reference force an ArithmeticExpression. (That shortcut does not work in ksh88.)

BASH and Korn shell arrays are also sparse. Elements may be added and deleted out of sequence.

•  # Bash/ksh
   arr[0]=0
   arr[1]=1
   arr[2]=2
   arr[42]="what was the question?"
   unset arr[2]
   echo "${arr[*]}"
   # prints 0 1 what was the question?

1. Loading values into an array

It's possible to assign multiple values to an array at once, but the syntax differs across shells.

•  # Bash
   array=(one two three four)
  
   # Korn
   set -A array -- one two three four

Bash also lets you initialize an array using a glob:

•  # Bash
   oggs=(*.ogg)

(see also NullGlob), or a substitution of any kind:

•  # Bash
   words=($sentence)
   set -f; O=$IFS IFS=$'\n' lines=($(< myfile)) IFS=$O; set +f
   letters=({a..z})    # Bash 3.0 or higher

When the arrname=(...) syntax is used, any substitutions inside the parentheses undergo WordSplitting according to the regular shell rules. Thus, in the second example above, if we want the lines of the input file to become individual array elements (even if they contain whitespace), we must set IFS appropriately (in this case: to a newline).

The set -f and set +f disable and re-enable glob expansion, respectively, so that a line like * will not be expanded into filenames. In some scripts, set -f may be in effect already, and therefore running set +f may be undesirable. This is something you must manage properly yourself; there is no easy or elegant way to "store" the glob expansion switch setting and restore it later. (And don't try to say parsing the output of set -o is easy, because it's not.)

If you're trying to populate an array with data from a stream, remember that in most shells, the subcommands of a pipeline are executed in subshells, so you might need to use something like this:

•  # Bash
   unset arr i
   while IFS= read -r 'arr[i++]'; do :; done < <(your command)
   if [[ ${arr[i-1]} = "" ]]; then unset 'arr[--i]'; fi

If you are trying to deal with records that might have embedded newlines, you might be using the NUL character ( \0 ) to delimit the records. In that case, you'll want to use the -d argument to read as well:

•  # Bash
   unset arr i
   while IFS= read -rd $'\0' 'arr[i++]'; do :; done < <(find . -name '*.ugly' -print0)
   if [[ ${arr[i-1]} = "" ]]; then unset 'arr[--i]'; fi

See ProcessSubstitution and FAQ #24 for more details on that syntax.

NOTE: it is necessary to quote the subscripted expression passed to read, so that the square brackets aren't interpreted as globs. This is also true for other non-keyword builtins that take a subscripted variable name, such as let and unset.

NOTE: if, as usual, there is a delimiter (newline and NUL) at the end of the input, the loops will add an empty array element (hence the subsequent unset). If the last array element was not empty, the input did not end in a delimiter. Alternatively, one can read into a temporary variable and set the array element from the temporary variable inside the loop.

greycat mentioned in IRC that he thought he recalled there being some reason not to use the 'while read arr[i++]' syntax directly, but he couldn't remember the specifics at the time. Perhaps it was one of these two issues.

If you wish to append data to an existing array, there are several approaches. The most flexible is to keep a separate index variable:

•  # Bash/ksh93
   arr[i++]="new item"

If you don't want to keep an index variable, but you happen to know that your array is not sparse, then you can use the highest existing index:

•  # Bash/ksh
   # This will FAIL if the array has holes (is sparse).
   arr[${#arr[*]}]="new item"

If you don't know whether your array is sparse or not, but you don't mind re-indexing the entire array (and also being very slow), then you can use:

•  # Bash
   arr=("${arr[@]}" "new item")
  
   # Ksh
   set -A arr -- "${arr[@]}" "new item"

If you're in bash 3.1 or higher, then you can use the += operator:

•  # Bash 3.1
   arr+=("new item")

For examples of using arrays to hold complex shell commands, see FAQ #50 and FAQ #40.

2. Retrieving values from an array

Using array elements en masse is one of the key features. In exactly the same way that "$@" is expanded for positional parameters, "${arr[@]}" is expanded to a list of words, one array element per word. For example,

•  # Korn/Bash
   for x in "${arr[@]}"; do
     echo "next element is '$x'"
   done

This works even if the elements contain whitespace. You always end up with the same number of words as you have array elements.

If one simply wants to dump the full array, one element per line, this is the simplest approach:

•  # Bash/ksh
   printf "%s\n" "${arr[@]}"

For more complex array-dumping, "${arr[*]}" will cause the elements to be concatenated together, with the first character of IFS (or a space if IFS isn't set) between them. As it happens, "$*" is expanded the same way for positional parameters.

•  # Bash
   arr=(x y z)
   IFS=/; echo "${arr[*]}"; unset IFS
   # prints x/y/z

BASH 3.0 added the ability to retrieve the list of index values in an array, rather than just iterating over the elements:

•  # Bash 3.0 or higher
   arr=(0 1 2 3) arr[42]='what was the question?'
   unset arr[2]
   echo ${!arr[*]}
   # prints 0 1 3 42

Bash's Parameter Expansions may be performed on array elements en masse as well:

•  # Bash
   arr=(abc def ghi jkl)
   echo "${arr[@]#?}"          # prints bc ef hi kl
   echo "${arr[@]/[aeiou]/}"   # prints bc df gh jkl

Parameter Expansion can also be used to extract elements from an array:

•  # Bash
   echo "${arr[@]:1:3}"        # three elements starting at #1 (second element)
   echo "${arr[@]:(-2)}"       # last two elements
   echo "${@:(-1)}"            # last positional parameter
   echo "${@:(-2):1}"          # second-to-last positional parameter

The @ array (the array of positional parameters) can be used just like any regularly named array.