1. How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?

Don't try to use "for". Use a while loop and the read command. Here is the basic template; there are many variations to discuss:

   1 while IFS= read -r line; do
   2   printf '%s\n' "$line"
   3 done < "$file"

line is a variable name, chosen by you. You can use any valid shell variable name(s) there; see field splitting below.

< "$file" redirects the loop's input from a file whose name is stored in a variable; see source selection below.

If you want to read lines from a file into an array, see FAQ 5.

1.1. Field splitting, whitespace trimming, and other input processing

The -r option to read prevents backslash interpretation (usually used as a backslash newline pair, to continue over multiple lines or to escape the delimiters). Without this option, any unescaped backslashes in the input will be discarded. You should almost always use the -r option with read.

The most common exception to this rule is when -e is used, which uses Readline to obtain the line from an interactive shell. In that case, tab completion will add backslashes to escape spaces and such, and you do not want them to be literally included in the variable. This would never be used when reading anything line-by-line, though, and -r should always be used when doing so.

By default, read modifies each line read, by removing all leading and trailing whitespace characters (spaces and tabs, if present in IFS). If that is not desired, the IFS variable may be cleared, as in the example above. If you want the trimming, leave IFS alone:

   1 # Leading/trailing whitespace trimming.
   2 while read -r line; do
   3   printf '%s\n' "$line"
   4 done < "$file"

If you want to operate on individual fields within each line, you may supply additional variables to read:

   1 # Input file has 3 columns separated by white space (space or tab characters only).
   2 while read -r first_name last_name phone; do
   3   # Only print the last name (second column)
   4   printf '%s\n' "$last_name"
   5 done < "$file"

If the field delimiters are not whitespace, you can set IFS (internal field separator):

   1 # Extract the username and its shell from /etc/passwd:
   2 while IFS=: read -r user pass uid gid gecos home shell; do
   3   printf '%s: %s\n' "$user" "$shell"
   4 done < /etc/passwd

For tab-delimited files, use IFS=$'\t' though beware that multiple tab characters in the input will be considered as one delimiter (and the Ksh93/Zsh IFS=$'\t\t' workaround won't work in Bash).

You do not necessarily need to know how many fields each line of input contains. If you supply more variables than there are fields, the extra variables will be empty. If you supply fewer, the last variable gets "all the rest" of the fields after the preceding ones are satisfied. For example,

   1 # Bash
   2 read -r first last junk <<< 'Bob Smith 123 Main Street Elk Grove Iowa 123-555-6789'
   3 
   4 # first will contain "Bob", and last will contain "Smith".
   5 # junk holds everything else.

Some people use the throwaway variable _ as a "junk variable" to ignore fields. It (or indeed any variable) can also be used more than once in a single read command, if we don't care what goes into it:

   1 # Bash
   2 read -r _ _ first middle last _ <<< "$record"
   3 
   4 # We skip the first two fields, then read the next three.
   5 # Remember, the final _ can absorb any number of fields.
   6 # It doesn't need to be repeated there.

Note that this usage of _ is only guaranteed to work in Bash. Many other shells use _ for other purposes that will at best cause this to not have the desired effect, and can break the script entirely. It is better to choose a unique variable that isn't used elsewhere in the script, even though _ is a common Bash convention.

If avoiding comments starting with # is desired, you can simply skip them inside the loop:

   1 # Bash
   2 while read -r line; do
   3   [[ $line = \#* ]] && continue
   4   printf '%s\n' "$line"
   5 done < "$file"

1.2. Input source selection

The redirection < "$file" tells the while loop to read from the file whose name is in the variable file. If you would prefer to use a literal pathname instead of a variable, you may do that as well. If your input source is the script's standard input, then you don't need any redirection at all.

If your input source is the contents of a variable/parameter, bash can iterate over its lines using a here string:

   1 while IFS= read -r line; do
   2   printf '%s\n' "$line"
   3 done <<< "$var"

The same can be done in any Bourne-type shell by using a "here document" (although read -r is POSIX, not Bourne):

   1 while IFS= read -r line; do
   2   printf '%s\n' "$line"
   3 done <<EOF
   4 $var
   5 EOF

One may also read from a command instead of a regular file:

   1 some command | while IFS= read -r line; do
   2   printf '%s\n' "$line"
   3 done

This method is especially useful for processing the output of find with a block of commands:

   1 find . -type f -print0 | while IFS= read -r -d '' file; do
   2     mv "$file" "${file// /_}"
   3 done

This reads one filename at a time from the find command and renames the file, replacing spaces with underscores.

Note the usage of -print0 in the find command, which uses NUL bytes as filename delimiters; and -d '' in the read command to instruct it to read all text into the file variable until it finds a NUL byte. By default, find and read delimit their input with newlines; however, since filenames can potentially contain newlines themselves, this default behaviour will split up those filenames at the newlines and cause the loop body to fail. Additionally it is necessary to set IFS to an empty string, because otherwise read would still strip leading and trailing whitespace. See FAQ #20 for more details.

Using a pipe to send find's output into a while loop places the loop in a SubShell, which means any state changes you make (changing variables, cd, opening and closing files, etc.) will be lost when the loop finishes. To avoid that, you may use a ProcessSubstitution:

   1 linecount=0
   2 
   3 while IFS= read -r line; do
   4   linecount=$((linecount + 1))
   5 done < <(some command)
   6 
   7 printf 'total lines: %d\n' "$linecount"

See FAQ 24 for more discussion.

1.3. My text files are broken! They lack their final newlines!

If there are some characters after the last line in the file (or to put it differently, if the last line is not terminated by a newline character), then read will read it but return false, leaving the broken partial line in the read variable(s). You can process this after the loop:

   1 # Emulate cat
   2 while IFS= read -r line; do
   3   printf '%s\n' "$line"
   4 done < "$file"
   5 [[ -n $line ]] && printf %s "$line"

Or:

   1 # This does not work:
   2 printf 'line 1\ntruncated line 2' | while read -r line; do
   3   echo $line
   4 done
   5 
   6 # This does not work either:
   7 printf 'line 1\ntruncated line 2' | while read -r line; do
   8   echo "$line"
   9 done
  10 [[ $line ]] && echo -n "$line"
  11 
  12 # This works:
  13 printf 'line 1\ntruncated line 2' | {
  14   while read -r line; do
  15     echo "$line"
  16   done
  17   [[ $line ]] && echo "$line"
  18 }

The first example, beyond missing the after-loop test, is also missing quotes. See Quotes or Arguments for an explanation why. The Arguments page is an especially important read.

For a discussion of why the second example above does not work as expected, see FAQ #24.

Alternatively, you can simply add a logical OR to the while test:

   1 while IFS= read -r line || [[ -n $line ]]; do
   2   printf '%s\n' "$line"
   3 done < "$file"
   4 
   5 printf 'line 1\ntruncated line 2' | while read -r line || [[ -n $line ]]; do
   6   echo "$line"
   7 done

1.4. How to keep other commands from "eating" the input

Some commands greedily eat up all available data on standard input. The examples above do not take precautions against such programs. For example,

   1 while read -r line; do
   2   cat > ignoredfile
   3   printf '%s\n' "$line"
   4 done < "$file"

will only print the contents of the first line, with the remaining contents going to "ignoredfile", as cat slurps up all available input.

One workaround is to use a numeric FileDescriptor rather than standard input:

   1 # Bash
   2 while IFS= read -r -u 9 line; do
   3   cat > ignoredfile
   4   printf '%s\n' "$line"
   5 done 9< "$file"
   6 
   7 # Note that read -u is not portable to every shell.
   8 # Use a redirect to ensure it works in any POSIX compliant shell:
   9 while IFS= read -r line <&9; do
  10   cat > ignoredfile
  11   printf '%s\n' "$line"
  12 done 9< "$file"

Or:

   1 exec 9< "$file"
   2 while IFS= read -r line <&9; do
   3   cat > ignoredfile
   4   printf '%s\n' "$line"
   5 done
   6 exec 9<&-

This example will wait for the user to type something into the file ignoredfile at each iteration instead of eating up the loop input.

You might need this, for example, with mencoder which will accept user input if there is any, but will continue silently if there isn't. Other commands that act this way include ssh and ffmpeg. Additional workarounds for this are discussed in FAQ #89.

CategoryShell CategoryBashguide

2. How can I store the return value and/or output of a command in a variable?

Well, that depends on whether you want to store the command's output (either stdout, or stdout + stderr) or its exit status (0 to 255, with 0 typically meaning "success").

If you want to capture the output, you use command substitution:

   1 output=$(command)      # stdout only; stderr remains uncaptured
   2 output=$(command 2>&1) # both stdout and stderr will be captured

If you want the exit status, you use the special parameter $? after running the command:

   1 command
   2 status=$?

If you want both:

   1 output=$(command)
   2 status=$?

The assignment to output has no effect on command's exit status, which is still in $?.

If you don't actually want to store the exit status, but simply want to take an action upon success or failure, just use if:

   1 if command; then
   2     printf "it succeeded\n"
   3 else
   4     printf "it failed\n"
   5 fi

Or if you want to capture stdout as well as taking action on success/failure, without explicitly storing or checking $?:

   1 if output=$(command); then
   2     printf "it succeeded\n"
   3     ...

If you don't understand the difference between standard output and standard error. here is a brief demonstration. A sane command writes the output that you request to standard output (stdout) and only writes errors to standard error (stderr). Like so:

$ dig +short A mywiki.wooledge.org
199.231.184.176
$ ip=$(dig +short A mywiki.wooledge.org)
$ echo "{$ip}"
{199.231.184.176}

$ ls no-such-file
ls: cannot access 'no-such-file': No such file or directory
$ output=$(ls no-such-file)
ls: cannot access 'no-such-file': No such file or directory
$ echo "{$output}"
{}

In the example above, dig wrote output to stdout, which was captured in the ip variable. ls encountered an error, so it did not write anything to stdout. It wrote to stderr, which was not captured (because we didn't use 2>&1). The error message appeared directly on the terminal instead.

Some commands are not well-written, however, and may write information to the wrong place. You must keep an eye out for such commands, and work around them when necessary. For example:

$ vers=$(python --version)
Python 2.7.13
$ echo "{$vers}"
{}

Even though we specifically asked for the version number, python wrote it to stderr. Thus, it appeared on the terminal, and was not captured in the vers variable. You'd need to use 2>&1 here.

What if you want the exit status of one command from a pipeline? If you want the last command's status, no problem -- it's in $? just like before. If you want some other command's status, use the PIPESTATUS array (BASH only. In the case of Zsh, it's lower-cased pipestatus). Say you want the exit status of grep in the following:

   1 grep foo somelogfile | head -n5
   2 status=${PIPESTATUS[0]}

Bash 3.0 added a pipefail option as well, which can be used if you simply want to take action upon failure of the grep:

   1 set -o pipefail
   2 if ! grep foo somelogfile | head -n5; then
   3     printf "uh oh\n"
   4 fi

Now, some trickier stuff. Let's say you want only the stderr, but not stdout. Well, then first you have to decide where you do want stdout to go:

   1 output=$(command 2>&1 >/dev/null)  # Save stderr, discard stdout.
   2 output=$(command 2>&1 >/dev/tty)   # Save stderr, send stdout to the terminal.
   3 output=$(command 3>&2 2>&1 1>&3-)  # Save stderr, send stdout to script's stderr.

Since the last example may seem a bit confusing, here is the explanation. First, keep in mind that 1>&3- is equivalent to 1>&3 3>&-. So it will be easier to analyse the following sequence: $(... 3>&2 2>&1 1>&3 3>&-)

A little note: operation n>&m- is sometimes called moving FD m to FD n.

This way what the script writes to FD 2 (normally stderr) will be written to stdout because of the second redirection. What the script writes to FD 1 (normally stdout) will be written to stderr because of the first and third redirections. Stdout and stderr got replaced. Done.

It's possible, although considerably harder, to let stdout "fall through" to wherever it would've gone if there hadn't been any redirection. This involves "saving" the current value of stdout, so that it can be used inside the command substitution:

   1 exec 3>&1                    # Save the place that stdout (1) points to.
   2 output=$(command 2>&1 1>&3)  # Run command.  stderr is captured.
   3 exec 3>&-                    # Close FD #3.
   4 
   5 # Or this alternative, which captures stderr, letting stdout through:
   6 { output=$(command 2>&1 1>&3-) ;} 3>&1

In the last example above, note that 1>&3- duplicates FD 3 and stores a copy in FD 1, and then closes FD 3. It could also be written 1>&3 3>&-.

What you cannot do is capture stdout in one variable, and stderr in another, using only FD redirections. You must use a temporary file (or a named pipe) to achieve that one.

Well, you can use a horrible hack like:

   1 cmd() { curl -s -v http://www.google.fr; }
   2 
   3 result=$(
   4     { stdout=$(cmd) ; } 2>&1
   5     printf "this line is the separator\n"
   6     printf "%s\n" "$stdout"
   7 )
   8 var_out=${result#*this line is the separator$'\n'}
   9 var_err=${result%$'\n'this line is the separator*}

Obviously, this is not robust, because either the standard output or the standard error of the command could contain whatever separator string you employ.

And if you want the exit code of your cmd (here a modification in the case of if the cmd stdout nothing)

   1 cmd() { curl -s -v http://www.google.fr; }
   2 
   3 result=$(
   4     { stdout=$(cmd); returncode=$?; } 2>&1
   5     printf "this is the separator"
   6     printf "%s\n" "$stdout"
   7     exit "$returncode"
   8 )
   9 returncode=$?
  10 
  11 var_out=${result#*this is the separator}
  12 var_err=${result%this is the separator*}

Note: the original question read, "How can I store the return value of a command in a variable?" This was, verbatim, an actual question asked in #bash, ambiguity and all.

CategoryShell

3. How can I sort or compare files based on some metadata attribute (newest / oldest modification time, size, etc)?

The tempting solution is to use ls to output sorted filenames and operate on the results using e.g. awk. As usual, the ls approach cannot be made robust and should never be used in scripts due in part to the possibility of arbitrary characters (including newlines) present in filenames. Therefore, we need some other way to compare file metadata.

The most common requirements are to get the most or least recently modified, or largest or smallest files in a directory. Bash and all ksh variants can compare modification times (mtime) using the -nt and -ot operators of the conditional expression compound command:

   1 unset -v latest
   2 for file in "$dir"/*; do
   3   [[ $file -nt $latest ]] && latest=$file
   4 done

Or to find the oldest:

   1 unset -v oldest
   2 for file in "$dir"/*; do
   3   [[ -z $oldest || $file -ot $oldest ]] && oldest=$file
   4 done

Keep in mind that mtime on directories is that of the most recently added, removed, or renamed file in that directory. Also note that -nt and -ot are not specified by POSIX test, but many shells such as dash include them anyway. No bourne-like shell has analogous operators for comparing by atime or ctime, so one would need external utilities for that; however, it's nearly impossible to either produce output which can be safely parsed, or handle said output in a shell without using nonstandard features on both ends.

If the sorting criteria are different from "oldest or newest file by mtime", then GNU find and GNU sort may be used together to produce a sorted list of filenames + timestamps, delimited by NUL characters. This will operate recursively by default. GNU find's -maxdepth operator can limit the search depth to 1 directory if needed. Here are a few possibilities, which can be modified as necessary to use atime or ctime, or to sort in reverse order:

   1 # GNU find + GNU sort (To the precision possible on the given OS, but returns only one result)
   2 IFS= read -r -d '' latest \
   3   < <(find "$dir" -type f -printf '%T@ %p\0' | sort -znr)
   4 latest=${latest#* }   # remove timestamp + space

   1 # GNU find (To the nearest 1s, using "find -printf" format (%Ts).)
   2 while IFS= read -rd '' time; do
   3   IFS= read -rd '' 'latest[time]'
   4 done < <(find "$dir" -type f -printf '%Ts\0%p\0')
   5 latest=${latest[-1]}

One disadvantage to these approaches is that the entire list is sorted, whereas simply iterating through the list to find the minimum or maximum timestamp (assuming we want just one file) would be faster. However, depending on the size of the job, the algorithmic disadvantage of sorting may be negligible in comparison to the overhead of using a shell.

   1 # GNU find
   2 unset -v latest time
   3 while IFS= read -r -d '' line; do
   4   t=${line%% *} t=${t%.*}   # truncate fractional seconds
   5   ((t > time)) && { latest=${line#* } time=$t; }
   6 done < <(find "$dir" -type f -printf '%T@ %p\0')

Similar usage patterns work well on many kinds of filesystem meta-data. This example gets the largest file in each subdirectory recursively. This is a common pattern for performing a calculation on a collection of files in each directory.

Readers who are asking this question in order to rotate their log files may wish to look into logrotate(1) instead, if their operating system provides it.

CategoryShell

4. How can I check whether a directory is empty or not? How do I check for any *.mpg files, or count how many there are?

In Bash, you can count files safely and easily with the nullglob and dotglob options (which change the behaviour of globbing), and an array:

   1 # Bash
   2 shopt -s nullglob dotglob
   3 files=(*)
   4 (( ${#files[*]} )) || echo directory is empty
   5 shopt -u nullglob dotglob

See ArithmeticExpression for explanations of arithmetic commands.

Of course, you can use any glob you like instead of *. E.g. *.mpg or /my/music/*.mpg works fine.

Bear in mind that you need read permission on the directory, or it will always appear empty.

Some people dislike nullglob because having unmatched globs vanish altogether confuses programs like ls. Mistyping ls *.zip as ls *.zpi may cause every file to be displayed (for such cases consider setting failglob). Setting nullglob in a SubShell avoids accidentally changing its setting in the rest of the shell, at the price of an extra fork(). If you'd like to avoid having to set and unset shell options, you can pour it all into a SubShell:

   1 # Bash
   2 if (shopt -s nullglob dotglob; f=(*); ((! ${#f[@]}))); then
   3     echo "The current directory is empty."
   4 fi

The other disadvantage of this approach (besides the extra fork()) is that the array is lost when the subshell exits. If you planned to use those filenames later, then they have to be retrieved all over again.

Both of these examples expand a glob and store the resulting filenames into an array, and then check whether the number of elements in the array is 0. If you actually want to see how many files there are, just print the array's size instead of checking whether it's 0:

   1 # Bash
   2 shopt -s nullglob dotglob
   3 files=(*)
   4 echo "The current directory contains ${#files[@]} things."

You can also avoid the nullglob if you're OK with putting a non-existing filename in the array should no files match (instead of an empty array):

   1 # Bash
   2 shopt -s dotglob
   3 files=(*)
   4 if [[ -e ${files[0]} || -L ${files[0]} ]]; then
   5     echo "The current directory is not empty.  It contains:"
   6     printf '%s\n' "${files[@]}"
   7 fi

Without nullglob, if there are no files in the directory, the glob will be added as the only element in the array. Since * is a valid filename, we can't simply check whether the array contains a literal *. So instead, we check whether the thing in the array exists as a file. The -L test is required because -e fails if the first file is a dangling symlink.

If you don't care how many matching files there are and don't want to store the results in an array, you can use bash's compgen command. Unfortunately, due to a bug, you need to use a hack to make it recognize dotglob:

   1 # Bash
   2 if (shopt -s dotglob; : *; compgen -G '*' >/dev/null); then
   3     echo "The current directory is not empty."
   4 else
   5     echo "The current directory is empty."
   6 fi

Or you can use an extended glob:

   1 # Bash
   2 # The subshell may be avoided by enabling extglob for the whole script.
   3 # Doing so should be safe.
   4 if (shopt -s extglob; compgen -G '@(*|.[!.]*|..?*)' >/dev/null); then
   5     echo "The current directory is not empty."
   6 else
   7     echo "The current directory is empty."
   8 fi

You may also use failglob:

   1 # Bash
   2 if ( shopt -s dotglob failglob; : ./* ) 2>/dev/null; then
   3     echo "The current directory is not empty."
   4 else
   5     echo "The current directory is empty."
   6 fi

But, if you use failglob, note that the subshell is required; the following code does not work because failglob will raise a shell error that will cause bash to stop running the current command (including the if command, any outer compound command, and the entire function that ran this code if it is part of a function), so this will only work in the true case, the else branch will never run:

   1 # BROKEN!
   2 shopt -s dotglob failglob
   3 if { : ./* ;} 2> /dev/null; then
   4     echo "The current directory is not empty."
   5 else
   6     echo "The current directory is empty."
   7 fi

If you really want to avoid using the subshell and want to set failglob globally, you can either "catch" the shell error using command eval, or you can write a function that expands the glob indirectly:

   1 shopt -s dotglob failglob
   2 if command eval ': ./*' 2> /dev/null; then
   3     echo "The current directory is not empty."
   4 else
   5     echo "The current directory is empty."
   6 fi
   7 # or
   8 shopt -s dotglob failglob
   9 any_match () { local IFS=; { : $@ ;} 2> /dev/null ;}
  10 if any_match './*'; then
  11     echo "The current directory is not empty."
  12 else
  13     echo "The current directory is empty."
  14 fi

If your script needs to run with various non-Bash shell implementations, you can try using an external program like python, perl, or find; or you can try one of these. Note the "magic 3 globs"¹ as POSIX does not have the dotglob option.

   1 # POSIX
   2 # Clobbers the positional parameters, so make sure you don't need them.
   3 set -- * .[!.]* ..?*
   4 for f in "$@"; do
   5   if test -e "$f" || test -L "$f"; then
   6     echo "directory is non-empty"
   7     break
   8   fi
   9 done

At this stage, the positional parameters have been loaded with the contents of the directory, and can be used for processing.

If you just want to count files:

   1 # POSIX
   2 n=0
   3 for f in * .[!.]* ..?*; do
   4   if test -e "$f" || test -L "$f"; then n=$((n+1)); fi
   5 done
   6 printf "There are %d files.\n" "$n"

In the Bourne shell, it's even worse, because there is no test -e or test -L:

   1 # Bourne
   2 # (Of course, the system must have printf(1).)
   3 if test "`printf '%s %s %s' .* *`" = '. .. *' && test ! -f '*'
   4 then
   5     echo "directory is empty"
   6 fi

Of course, that fails if * exists as something other than a plain file (such as a directory or FIFO). The absence of a -e test really hurts.

Here is another solution using find:

   1 # POSIX
   2 # Print a single `.' for each file and count the number of characters printed.
   3 # This one will recurse.  If that is not desired, see below.
   4 n=$(find . -type f -exec printf %.0s. {} + | wc -m)
   5 printf "There are %d files.\n" "$n"

If you want it not to recurse, then you need to tell find not to recurse into directories. This gets really tricky and ugly. GNU find has a -maxdepth option to do it. With standard POSIX find, you're stuck with -prune. This is left as an exercise for the reader.

Never try to parse ls output. Even ls -A solutions can break (e.g. on HP-UX, if you are root, ls -A does the exact opposite of what it does if you're not root -- and no, I can't make up something that incredibly stupid).

In fact, one may wish to avoid the direct question altogether. Usually people want to know whether a directory is empty because they want to do something involving the files therein, etc. Look to the larger question. For example, one of these find-based examples may be an appropriate solution:

   1 # Bourne / POSIX
   2 find "$somedir" -type f -exec echo Found unexpected file {} \;
   3 find "$somedir" -prune -empty -exec printf '%s is empty.\n' {} \;  # GNU/BSD
   4 find "$somedir" -type d -empty -exec cp /my/configfile {} \;   # GNU/BSD

Most commonly, all that's really needed is something like this:

   1 # Bourne / POSIX
   2 for f in ./*.mpg; do
   3     test -f "$f" || continue
   4     mympgviewer "$f"
   5 done

In other words, the person asking the question may have thought an explicit empty-directory test was needed to avoid an error message like mympgviewer: ./*.mpg: No such file or directory when in fact no such test is required.

Support for a nullglob-like feature is inconsistent. In ksh93 it can be done on a per-pattern basis by prefixing with ~(N)²:

   1 # ksh93
   2 for f in ~(N)*; do
   3     ....
   4 done

CategoryShell

5. How can I use array variables?

This answer assumes you have a basic understanding of what arrays are. If you're new to this kind of programming, you may wish to start with the guide's explanation. This page is more thorough. See links at the bottom for more resources.

5.1. Intro

One-dimensional integer-indexed arrays are implemented by Bash, Zsh, and most KornShell varieties including AT&T ksh88 or later, mksh, and pdksh. Arrays are not specified by POSIX and not available in legacy or minimalist shells such as BourneShell and Dash. The POSIX-compatible shells that do feature arrays mostly agree on their basic principles, but there are some significant differences in the details. Advanced users of multiple shells should be sure to research the specifics. Ksh93, Zsh, and Bash 4.0 additionally have Associative Arrays (see also FAQ 6). This article focuses on indexed arrays as they are the most common type.

Basic syntax summary (for bash, math indexed arrays):

Here is a typical usage pattern featuring an array named host:

   1 # Bash
   2 
   3 # Assign the values "mickey", "minnie", and "goofy" to sequential indexes starting with zero.
   4 host=(mickey minnie goofy)
   5 
   6 # Iterate over the indexes of "host".
   7 for idx in "${!host[@]}"; do
   8     printf 'Host number %d is %s\n' "$idx" "${host[idx]}"
   9 done

"${!host[@]}" expands to the indices of of the host array, each as a separate word.

Indexed arrays are sparse, and elements may be inserted and deleted out of sequence.

   1 # Bash/ksh
   2 
   3 # Simple assignment syntax.
   4 arr[0]=0
   5 arr[2]=2
   6 arr[1]=1
   7 arr[42]='what was the question?'
   8 
   9 # Unset the second element of "arr"
  10 unset -v 'arr[2]'
  11 
  12 # Concatenate the values, to a single argument separated by spaces, and echo the result.
  13 echo "${arr[*]}"
  14 # outputs: "0 1 what was the question?"

It is good practice to write your code in such a way that it can handle sparse arrays, even if you think you can guarantee that there will never be any "holes". Only treat arrays as "lists" if you're certain, and the savings in complexity is significant enough for it to be justified.

5.2. Loading values into an array

Assigning one element at a time is simple, and portable:

   1 # Bash/ksh
   2 arr[0]=0
   3 arr[42]='the answer'

It's possible to assign multiple values to an array at once, but the syntax differs across shells. Bash supports only the arrName=(args...) syntax. ksh88 supports only the set -A arrName -- args... syntax. ksh93, mksh, and zsh support both. There are subtle differences in both methods between all of these shells if you look closely.

   1 # Bash, ksh93, mksh, zsh
   2 array=(zero one two three four)

   1 # ksh88/93, mksh, zsh
   2 set -A array -- zero one two three four

When initializing in this way, the first index will be 0 unless a different index is specified.

With compound assignment, the space between the parentheses is evaluated in the same way as the arguments to a command, including pathname expansion and WordSplitting. Any type of expansion or substitution may be used. All the usual quoting rules apply within.

   1 # Bash/ksh93
   2 oggs=(*.ogg)

With ksh88-style assignment using set, the arguments are just ordinary arguments to a command.

   1 # Korn
   2 set -A oggs -- *.ogg

   1 # Bash (brace expansion requires 3.0 or higher)
   2 homeDirs=(~{,root}) # brace expansion occurs in a different order in ksh, so this is bash-only.
   3 letters=({a..z})    # Not all shells with sequence-expansion can use letters.

   1 # Korn
   2 set -A args -- "$@"

5.2.1. Loading lines from a file or stream

In bash 4, the mapfile command (also known as readarray) accomplishes this:

   1 # Bash 4
   2 mapfile -t lines <myfile
   3 
   4 # or
   5 mapfile -t lines < <(some command)

See ProcessSubstitution and FAQ #24 for more details on the <(...) syntax.

mapfile handles blank lines by inserting them as empty array elements, and (with -t) also silently appends a missing final newline if the input stream lacks one. These can be problematic when reading data in other ways (see the next section). mapfile in bash 4.0 through 4.3 does have one serious drawback: it can only handle newlines as line terminators. Bash 4.4 adds the -d option to supply a different line delimiter.

When mapfile isn't available, we have to work very hard to try to duplicate it. There are a great number of ways to almost get it right, but many of them fail in subtle ways.

The following examples will duplicate most of mapfile's basic functionality in older shells. You can skip all of these alternative examples if you have bash 4.

   1 # Alternative: Bash 3.1, Ksh93, mksh
   2 unset -v lines
   3 while IFS= read -r; do
   4     lines+=("$REPLY")
   5 done <file
   6 [[ $REPLY ]] && lines+=("$REPLY")

The += operator, when used together with parentheses, appends the element to one greater than the current highest numbered index in the array.

   1 # Alternative: ksh88
   2 # Ksh88 doesn't support pre/post increment/decrement. mksh and others do.
   3 i=0
   4 unset -v lines
   5 while IFS= read -r; do
   6     lines[i+=1,$i]=$REPLY     # Mimics lines[i++]=$REPLY
   7 done <file
   8 [[ $REPLY ]] && lines[i]=$REPLY

The square brackets create a math context. The result of the expression is the index used for assignment.

5.2.1.1. Handling newlines (or lack thereof) at the end of a file

read returns false when it reads the last line of a file. This presents a problem: if the file contains a trailing newline, then read will be false when reading/assigning that final line, otherwise, it will be false when reading/assigning the last line of data. Without a special check for these cases, no matter what logic is used, you will always end up either with an extra blank element in the resulting array, or a missing final element.

To be clear - text files should contain a newline as the last character in the file. Newlines are added to the ends of files by most text editors, and also by Here documents and Here strings. Most of the time, this is only an issue when reading output from pipes or process substitutions, or from "broken" text files created with broken or misconfigured tools. Let's look at some examples.

This approach reads the elements one by one, using a loop.

   1 # Doesn't work correctly!
   2 unset -v arr i
   3 while IFS= read -r 'arr[i++]'; do
   4     :
   5 done < <(printf '%s\n' {a..d})

Unfortunately, if the file or input stream contains a trailing newline, a blank element is added at the end of the array, because the read -r arr[i++] is executed one extra time after the last line containing text before returning false.

   1 # Still doesn't work correctly!
   2 unset -v arr i
   3 while read -r; do
   4     arr[i++]=$REPLY
   5 done < <(printf %s {a..c}$'\n' d)

The square brackets create a math context. Inside them, i++ works as a C programmer would expect (in all but ksh88).

This approach fails in the reverse case - it correctly handles blank lines and inputs terminated with a newline, but fails to record the last line of input. If the file or stream is missing its final newline. So we need to handle that case specially:

   1 # Alternative: Bash, ksh93, mksh
   2 unset -v arr i
   3 while IFS= read -r; do
   4     arr[i++]=$REPLY
   5 done <file
   6 [[ $REPLY ]] && arr[i++]=$REPLY # Append unterminated data line, if there was one.

This is very close to the "final solution" we gave earlier -- handling both blank lines inside the file, and an unterminated final line. The null IFS is used to prevent read from stripping possible whitespace from the beginning and end of lines, in the event you wish to preserve them.

Another workaround is to remove the empty element after the loop:

   1 # Alternative: Bash
   2 unset -v arr i
   3 while IFS= read -r 'arr[i++]'; do
   4     :
   5 done <file
   6 
   7 # Remove trailing empty element, if any.
   8 [[ ${arr[i-1]} ]] || unset -v 'arr[--i]'

Whether you prefer to read too many and then have to remove one, or read too few and then have to add one, is a personal choice.

NOTE: it is necessary to quote the 'arr[i++]' passed to read, so that the square brackets aren't interpreted as globs. This is also true for other non-keyword builtins that take a subscripted variable name, such as let and unset.

5.2.1.2. Other methods

Sometimes stripping blank lines actually is desirable, or you may know that the input will always be newline delimited, such as input generated internally by your script. It is possible in some shells to use the -d flag to set read's line delimiter to null, then abuse the -a or -A (depending on the shell) flag normally used for reading the fields of a line into an array for reading lines. Effectively, the entire input is treated as a single line, and the fields are newline-delimited.

   1 # Bash 4
   2 IFS=$'\n' read -rd '' -a lines <file

   1 # mksh, zsh
   2 IFS=$'\n' read -rd '' -A lines <file

5.2.1.3. Don't read lines with for!

Never read lines using for..in loops! Relying on IFS WordSplitting causes issues if you have repeated whitespace delimiters, because they will be consolidated. It is not possible to preserve blank lines by having them stored as empty array elements this way. Even worse, special globbing characters will be expanded without going to lengths to disable and then re-enable it. Just never use this approach - it is problematic, the workarounds are all ugly, and not all problems are solvable.

5.2.2. Reading NUL-delimited streams

If you are trying to deal with records that might have embedded newlines, you will be using an alternative delimiter such as the NUL character ( \0 ) to separate the records. In bash 4.4, you can simply use mapfile -t -d '':

   1 # Bash 4.4
   2 mapfile -t -d '' files < <(find . -name '*.ugly' -print0)

Otherwise, you'll need to use the -d argument to read inside a loop:

   1 # Bash
   2 while read -rd ''; do
   3     arr[i++]=$REPLY
   4 done < <(find . -name '*.ugly' -print0)
   5 
   6 # or (bash 3.1 and up)
   7 while read -rd ''; do
   8     arr+=("$REPLY")
   9 done < <(find . -name '*.ugly' -print0)

read -d '' tells Bash to keep reading until a NUL byte instead of until a newline. This isn't certain to work in all shells with a -d feature.

If you choose to give a variable name to read instead of using REPLY then also be sure to set IFS= for the read command, to avoid trimming leading/trailing IFS whitespace.

5.2.3. Appending to an existing array

As previously mentioned, arrays are sparse - that is, numerically adjacent indexes are not guaranteed to be occupied by a value. This confuses what it means to "append" to an existing array. There are several approaches.

If you've been keeping track of the highest-numbered index with a variable (for example, as a side-effect of populating an array in a loop), and can guarantee it's correct, you can just use it and continue to ensure it remains in-sync.

   1 # Bash/ksh93
   2 arr[++i]="new item"

If you don't want to keep an index variable, but happen to know that your array is not sparse, then you can use the number of elements to calculate the offset (not recommended):

   1 # Bash/ksh
   2 # This will FAIL if the array has holes (is sparse).
   3 arr[${#arr[@]}]="new item"

If you don't know whether your array is sparse or not, but don't mind re-indexing the entire array (very inefficient), then you can use:

   1 # Bash
   2 arr=("${arr[@]}" "new item")
   3 
   4 # Ksh
   5 set -A arr -- "${arr[@]}" "new item"

If you're in bash 3.1 or higher, then you can use the += operator:

   1 # Bash 3.1, ksh93, mksh, zsh
   2 arr+=(item 'another item')

NOTE: the parentheses are required, just as when assigning to an array. Otherwise you will end up appending to ${arr[0]} which $arr is a synonym for. If your shell supports this type of appending, it is the preferred method.

For examples of using arrays to hold complex shell commands, see FAQ #50 and FAQ #40.

5.3. Retrieving values from an array

${#arr[@]} or ${#arr[*]} expand to the number of elements in an array:

   1 # Bash
   2 shopt -s nullglob
   3 oggs=(*.ogg)
   4 echo "There are ${#oggs[@]} Ogg files."

Single elements are retrieved by index:

   1 echo "${foo[0]} - ${bar[j+1]}"

The square brackets are a math context. Within an arithmetic context, variables, including arrays, can be referenced by name. For example, in the expansion:

   1 ${arr[x[3+arr[2]]]}

arr's index will be the value from the array x whose index is 3 plus the value of arr[2].

Using array elements en masse is one of the key features of shell arrays. In exactly the same way that "$@" is expanded for positional parameters, "${arr[@]}" is expanded to a list of words, one array element per word. For example,

   1 # Korn/Bash
   2 for x in "${arr[@]}"; do
   3   echo "next element is '$x'"
   4 done

This works even if the elements contain whitespace. You always end up with the same number of words as you have array elements.

If one simply wants to dump the full array, one element per line, this is the simplest approach:

   1 # Bash/ksh
   2 printf "%s\n" "${arr[@]}"

For slightly more complex array-dumping, "${arr[*]}" will cause the elements to be concatenated together, with the first character of IFS (or a space if IFS isn't set) between them. As it happens, "$*" is expanded the same way for positional parameters.

   1 # Bash
   2 arr=(x y z)
   3 IFS=/; echo "${arr[*]}"; unset -v IFS
   4 # prints x/y/z

Unfortunately, you can't put multiple characters in between array elements using that syntax. You would have to do something like this instead:

   1 # Bash/ksh
   2 arr=(x y z)
   3 tmp=$(printf "%s<=>" "${arr[@]}")
   4 echo "${tmp%<=>}"    # Remove the extra <=> from the end.
   5 # prints x<=>y<=>z

Or using array slicing, described in the next section.

   1 # Bash/ksh
   2 typeset -a a=([0]=x [5]=y [10]=z)
   3 printf '%s<=>' "${a[@]::${#a[@]}-1}"
   4 printf '%s\n' "${a[@]:(-1)}"

This also shows how sparse arrays can be assigned multiple elements at once. Note using the arr=([key]=value ...) notation differs between shells. In ksh93, this syntax gives you an associative array by default unless you specify otherwise, and using it requires that every value be explicitly given an index, unlike bash, where omitted indexes begin at the previous index. This example was written in a way that's compatible between the two.

BASH 3.0 added the ability to retrieve the list of index values in an array:

   1 # Bash 3.0 or higher
   2 arr=(0 1 2 3) arr[42]='what was the question?'
   3 unset -v 'arr[2]'
   4 echo "${!arr[@]}"
   5 # prints 0 1 3 42

Retrieving the indices is extremely important for certain kinds of tasks, such as maintaining parallel arrays with the same indices (a cheap way to mimic having an array of structs in a language with no struct):

   1 # Bash 3.0 or higher
   2 unset -v file title artist i
   3 for f in ./*.mp3; do
   4   file[i]=$f
   5   title[i]=$(mp3info -p %t "$f")
   6   artist[i++]=$(mp3info -p %a "$f")
   7 done
   8 
   9 # Later, iterate over every song.
  10 # This works even if the arrays are sparse, just so long as they all have
  11 # the SAME holes.
  12 for i in "${!file[@]}"; do
  13   echo "${file[i]} is ${title[i]} by ${artist[i]}"
  14 done

5.3.1. Retrieving with modifications

Bash's Parameter Expansions may be performed on array elements en masse:

   1 # Bash
   2 arr=(abc def ghi jkl)
   3 echo "${arr[@]#?}"          # prints bc ef hi kl
   4 echo "${arr[@]/[aeiou]/}"   # prints bc df gh jkl

Parameter Expansion can also be used to extract sub-lists of elements from an array. Some people call this slicing:

   1 # Bash
   2 echo "${arr[@]:1:3}"        # three elements starting at #1 (second element)
   3 echo "${arr[@]:(-2)}"       # last two elements

The same goes for positional parameters

   1 set -- foo bar baz
   2 echo "${@:(-1)}"            # last positional parameter baz
   3 echo "${@:(-2):1}"          # second-to-last positional parameter bar

5.4. Using @ as a pseudo-array

As we see above, the @ array (the array of positional parameters) can be used almost like a regularly named array. This is the only array available for use in POSIX or Bourne shells. It has certain limitations: you cannot individually set or unset single elements, and it cannot be sparse. Nevertheless, it still makes certain POSIX shell tasks possible that would otherwise require external tools:

   1 # POSIX
   2 set -- *.mp3
   3 if [ -e "$1" ] || [ -L "$1" ]; then
   4   echo "there are $# MP3 files"
   5 else
   6   echo "there are 0 MP3 files"
   7 fi

   1 # POSIX
   2 ...
   3 # Add an option to our dynamically generated list of options
   4 set -- "$@" -f "$somefile"
   5 ...
   6 foocommand "$@"

(Compare to FAQ #50's dynamically generated commands using named arrays.)

6. See Also

• Bash-hackers array documentation

• BashGuide/Arrays

• BashSheet Array reference

• BashFAQ 6 - explaining associative arrays

CategoryShell

7. How can I use variable variables (indirect variables, pointers, references) or associative arrays?

This is a complex page, because it's a complex topic. It's been divided into roughly three parts: associative arrays, evaluating indirect variables, and assigning indirect variables. There are discussions of programming issues and concepts scattered throughout.

7.1. Associative Arrays

We introduce associative arrays first, because we observe that inexperienced programmers often conjure solutions to problems that would most typically utilize associative arrays by attempting to dynamically generate variables in a Hungarian notation scheme (in order to coerce the symbol table hashing function into resolving user-defined associative mappings).

An associative array is an unordered collection of key-value pairs. A value may be retrieved by supplying its corresponding key. Since strings are the only datatype most shells understand, associative arrays map strings to strings, unlike indexed arrays, which map integers to strings. Associative arrays exist in AWK as "associative arrays", in Perl as "hashes", in Tcl as "arrays", in Python and C# as "dictionaries", and in Java as a "Map", and in C++11 STL as std::unordered_map.

   1 # Bash 4 / ksh93
   2 
   3 typeset -A homedir    # Declare associative array
   4 homedir=(             # Compound assignment
   5     [jim]=/home/jim
   6     [silvia]=/home/silvia
   7     [alex]=/home/alex
   8 )
   9 
  10 homedir[ormaaj]=/home/ormaaj # Ordinary assignment adds another single element
  11 
  12 for user in "${!homedir[@]}"; do   # Enumerate all indices (user names)
  13     printf 'Home directory of user %q is: %q\n' "$user" "${homedir[$user]}"
  14 done

Prior to Bash 4 or if you can't use ksh93, your options are limited. Either move to another interpreter (awk, perl, python, ruby, tcl, ...) or re-evaluate your problem to simplify it. There are certain tasks for which associative arrays are a powerful and completely appropriate tool. There are others for which they are overkill, or simply unsuitable.

Suppose we have several subservient hosts with slightly different configuration, and that we want to ssh to each one and run slightly different commands. One way we could set it up would be to hard-code a bunch of ssh commands in per-hostname functions in a single script and just run them in series or in parallel. (Don't reject this out of hand! Simple is good.) Another way would be to store each group of commands as an element of an associative array keyed by the hostname:

   1 declare -A commands
   2 commands=(
   3   [host1]="mvn clean install && cd webapp && mvn jetty:run"
   4   [host2]="..."
   5 )
   6 
   7 for host in "${!commands[@]}"; do
   8     ssh -- "$host" "${commands[$host]}"
   9 done

This is the kind of approach we'd expect in a high-level language, where we can store hierarchical information in advanced data structures. The difficulty here is that we really want each element of the associative array to be a list or another array of command strings. But the shell simply doesn't permit that kind of data structure.

So, often it pays to step back and think in terms of shells rather than other programming languages. Aren't we just running a script on a remote host? Then why don't we just store the configuration sets as scripts? Then it's simple:

   1 # A series of conf files named for the hosts we need to run our commands on:
   2 for conf in /etc/myapp/*; do
   3     host=${conf##*/}
   4     ssh -- "$host" bash < "$conf"
   5 done
   6 
   7 # /etc/myapp/hostname is just a script:
   8 mvn clean install &&
   9 cd ./webapp &&
  10 mvn jetty:run

Now we've removed the need for associative arrays, and also the need to maintain a bunch of extremely horrible quoting issues. It is also easy to parallelize using GNU Parallel:

   1 parallel ssh -- {/} bash "<" {} ::: /etc/myapp/*

7.1.1. Associative array hacks in older shells

Before you think of using eval to mimic associative arrays in an older shell (probably by creating a set of variable names like homedir_alex), try to think of a simpler or completely different approach that you could use instead. If this hack still seems to be the best thing to do, consider the following disadvantages:

1. It's really hard to read, to keep track of, and to maintain.

2. The variable names must be a single line and match the RegularExpression ^[a-zA-Z_][a-zA-Z_0-9]*$ -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames, for instance -- consider a user named hong-hu. A dash '-' cannot be part of a variable name, so the entire attempt to make a variable named homedir_hong-hu is doomed from the start.

3. Quoting is hard to get right. If a content string (not a variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it through both shell parsings. And that's just for constants, known at the time you write the program. (Bash's printf %q helps, but nothing analogous is available in POSIX shells.)

4. If the program handles unsanitized user input, it can be VERY dangerous!

Read BashGuide/Arrays or BashFAQ/005 for a more in-depth description and examples of how to use arrays in Bash.

If you need an associative array but your shell doesn't support them, please consider using AWK instead.

7.2. Indirection

7.2.1. Think before using indirection

Putting variable names or any other bash syntax inside parameters is frequently done incorrectly and in inappropriate situations to solve problems that have better solutions. It violates the separation between code and data, and as such puts you on a slippery slope toward bugs and security issues. Indirection can make your code less transparent and harder to follow.

Normally, in bash scripting, you won't need indirect references at all. Generally, people look at this for a solution when they don't understand or know about Bash Arrays or haven't fully considered other Bash features such as functions.

7.2.2. Evaluating indirect/reference variables

BASH allows for expanding parameters indirectly -- that is, one variable may contain the name of another variable. Name reference variables are the preferred method for performing variable indirection. Older versions of Bash could also use a ! prefix operator in parameter expansions for variable indirection. Namerefs should be used unless portability to older bash versions is required. No other shell uses ${!variable} for indirection and there are problems relating to use of that syntax for this purpose. It is also less flexible.

   1 # Bash
   2 realvariable=contents
   3 ref=realvariable
   4 printf '%s\n' "${!ref}"   # prints the contents of the real variable

KornShell (ksh93) has a completely different, more powerful syntax -- the nameref command (also known as typeset -n):

   1 # ksh93 / mksh / Bash 4.3
   2 realvariable=contents
   3 typeset -n ref=realvariable
   4 printf '%s\n' "${!ref} = $ref"      # prints the name and contents of the real variable

Zsh allows you to access a parameter indirectly with the parameter expansion flag P:

   1 # zsh
   2 realvariable=contents
   3 ref=realvariable
   4 echo ${(P)ref}   # prints the contents of the real variable

Unfortunately, for shells other than Bash, ksh93, and zsh there is no syntax for evaluating a referenced variable. You would have to use eval, which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.

It's difficult to imagine a practical use for this that wouldn't be just as easily performed by using an associative array. But people ask it all the time (it is genuinely a frequently asked question).

ksh93's nameref allows us to work with references to arrays, as well as regular scalar variables. For example,

   1 # ksh93 / mksh / bash
   2 function myfunc {
   3   typeset -n ref=$1
   4   printf 'Array %s has %d elements.\n' "${!ref}" "${#ref[@]}"
   5 }
   6 
   7 realarray=(...)
   8 myfunc realarray

zsh's ability to nest parameter expansions allow for referencing arrays too:

   1 # zsh
   2 myfunc() {
   3  local ref=$1
   4  echo "array $1 has ${#${(@P)ref}} elements"
   5 }
   6 realarray=(...)
   7 myfunc realarray

We are not aware of any trick that can duplicate that functionality in POSIX or Bourne shells without eval, which can be difficult to do securely. Older versions of Bash can almost do it -- some indirect array tricks work, and others do not, and we do not know whether the syntax involved will remain stable in future releases. So, consider this a use at your own risk hack.

   1 # Bash -- trick #1.  Works in bash 2 and up, and ksh93v+ (when invoked as bash)
   2 realarray=(...) ref=realarray; index=2
   3 tmp=${ref}[index]
   4 echo "${!tmp}"            # gives array element [2]

   1 # Bash -- trick #2.  Seems to work in bash 3 and up.
   2 # Can't be combined with special expansions until 4.3. e.g. "${!tmp##*/}"
   3 # Does NOT work in bash 2.05b -- Expands to one word instead of three in bash 2.
   4 tmp=${ref}[@]
   5 printf '<%s> ' "${!tmp}"; echo    # Iterate whole array as one word per element.

It is not possible to retrieve array indices directly using the Bash ${!var} indirect expansion.

7.2.3. Assigning indirect/reference variables

Sometimes you'd like to "point" from one variable to another, for purposes of writing information to a dynamically configurable place. Typically this happens when you're trying to write a "reusable" function or library, and you want it to put its output in a variable of the caller's choice instead of the function's choice. (Various traits of Bash make safe reusability of Bash functions difficult at best, so this is something that should not happen often.)

Assigning a value "through" a reference (I'm going to use "ref" from now on) is more widely possible, but the means of doing so are usually extremely shell-specific. All shells with the sole exception of AT&T ksh93 lack real reference variables or pointers. Indirection can only be achieved by indirectly evaluating variable names. IOW, you can never have a real unambiguous reference to an object in memory, the best you can do is use the name of a variable to try simulating the effect. Therefore, you must control the value of the ref and ensure side-effects such as globbing, user-input, and conflicting local parameters can't affect parameter names. Names must either be deterministic or validated in a way that makes certain guarantees. If an end user can populate the ref variable with arbitrary strings, the result can be unexpected code injection. We'll show an example of this at the end.

In ksh93, we can use nameref again:

   1 # ksh93/mksh/Bash 4.3
   2 typeset -n ref=realvariable
   3 ref=contents
   4 # realvariable now contains the string "contents"

In zsh, using parameter expansions ::= and expansion flags P:

   1 # zsh
   2 ref=realvariable
   3 : ${(P)ref::=contents}
   4 # redefines realvariable unconditionally to the string "contents"

In Bash, if you only want to assign a single line to the variable, you can use read and Bash's here string syntax:

   1 # Bash/ksh93/mksh/zsh
   2 ref=realvariable
   3 IFS= read -r -- "$ref" <<<"contents"
   4 # realvariable now contains the string "contents"

If you need to assign multiline values, keep reading.

A similar trick works for Bash array variables too:

   1 # Bash
   2 aref=realarray
   3 IFS=' ' read -d '' -ra "$aref" <<<'words go into array elements'
   4 
   5 # ksh93/mksh/zsh
   6 aref=realarray
   7 IFS=' ' read -d '' -rA "$aref" <<<'words go into array elements'

IFS is used to delimit words, so you may or may not need to set that. Also note that the read command will return failure because there is no terminating NUL byte for the -d '' to catch. Be prepared to ignore that failure.

Another trick is to use Bash's printf -v (only available in recent versions):

   1 # Bash 3.1 or higher ONLY. Array assignments require 4.2 or higher.
   2 ref=realvariable
   3 printf -v "$ref" %s "contents"

You can use all of printf's formatting capabilities. This trick also permits any string content, including embedded and trailing newlines.

Yet another trick is Korn shell's typeset or Bash's declare. The details of typeset vary greatly between shells, but can be used in compatible ways in limited situations. Both of them cause a variable to become locally scoped to a function, if used inside a function; but if used outside all functions, they can operate on global variables.

   1 # Bash/ksh (any)/zsh
   2 typeset -- "${ref}=contents"
   3 
   4 # Bash
   5 declare -- "${ref}=contents"

Bash 4.2 adds declare -g which assigns variables to the global scope from any context.

There is very little advantage to typeset or declare over eval for scalar assignments, but many drawbacks. typeset cannot be made to not affect the scope of the assigned variable. This trick does preserve the exact contents, like eval, if correctly escaped.

You must still be careful about what is on the left-hand side of the assignment. Inside square brackets, expansions are still performed; thus, with a tainted ref, declare or printf -v can be just as dangerous as eval:

   1 # Bash:
   2 ref='x[$(touch evilfile)0]'
   3 ls -l evilfile   # No such file or directory
   4 declare "${ref}=value"
   5 ls -l evilfile   # It exists now!
   6 rm evilfile # Now it's gone.
   7 printf -v "$ref" %s "value"
   8 ls -l evilfile   # It came back!