<> == How can I use variable variables (indirect variables, pointers, references) or associative arrays? == This is a complex page, because it's a complex topic. It's been divided into roughly four parts: associative arrays, name references, evaluating indirect variables, and assigning indirect variables. There are discussions of programming issues and concepts scattered throughout. <> === Associative Arrays === We introduce associative arrays first, because we observe that inexperienced programmers often conjure arcane solutions to problems that would be solved more cleanly with associative arrays. An [[https://en.wikipedia.org/wiki/Associative_array|associative array]] is an unordered collection of key-value pairs. A value may be retrieved by supplying its corresponding key. Since strings are the only datatype most shells understand, associative arrays map strings to strings, unlike indexed arrays, which map integers to strings. Associative arrays exist in AWK as "associative arrays", in Perl as "hashes", in Tcl as "arrays", in Python and C# as "dictionaries", in Java as a "Map", and in C++11 STL as `std::unordered_map`. {{{#!highlight bash # Bash 4 / ksh93 typeset -A homedir # Declare associative array homedir=( # Compound assignment [jim]=/home/jim [silvia]=/u/silvia [alex]=/net/home/alex ) homedir[ormaaj]=/home/ormaaj # Ordinary assignment adds another single element for user in "${!homedir[@]}"; do # Enumerate all indices (user names) printf 'Home directory of user %q is: %q\n' "$user" "${homedir[$user]}" done }}} Prior to Bash 4 or if you can't use ksh93, your options are limited. Either move to another interpreter (awk, perl, python, ruby, tcl, ...) or re-evaluate your problem to ''simplify it''. There are certain tasks for which associative arrays are a powerful and completely appropriate tool. There are others for which they are overkill, or simply unsuitable. Suppose we have several remote hosts with slightly different configuration, and that we want to ssh to each one and run slightly different commands. One way we could set it up would be to hard-code a bunch of ssh commands in per-hostname functions in a single script and just run them in series or in parallel. (Don't reject this out of hand! Simple is good.) Another way would be to store each group of commands as an element of an associative array keyed by the hostname: {{{#!highlight bash declare -A commands commands=( [host1]="mvn clean install && cd webapp && mvn jetty:run" [host2]="..." ) for host in "${!commands[@]}"; do ssh -- "$host" "${commands[$host]}" done }}} This solution works, because we're encoding a very short shell script in a string, and storing it as an array element. When we call ssh, it passes the string directly to the remote host, where a shell evaluates it and executes it. But what if the scripts were much longer, or more complicated? If we want to get ''fancy'' and store each sub-command (`cd webapp` for example) as an element of a list, and then have each hostname map to a list of sub-commands, we'd quickly find that we can't do that in a shell. That's the kind of approach we'd expect in a high-level language, where we can store hierarchical information in advanced data structures. We want each element of the associative array to be a ''list'' or ''another array'' of command strings. But the shell simply doesn't offer that kind of data structure. So, often it pays to step back and ''think in terms of shells'' rather than other programming languages. Aren't we just running a script on a remote host? Then why don't we just store the configuration sets ''as scripts''? Then it's simple: {{{#!highlight bash # A series of conf files named for the hosts we need to run our commands on: for conf in /etc/myapp/*; do host=${conf##*/} ssh -- "$host" bash < "$conf" done # /etc/myapp/hostname is just a script: mvn clean install && cd ./webapp && mvn jetty:run }}} Now we've removed the need for associative arrays, and also the need to maintain a bunch of extremely horrible quoting issues. It is also easy to parallelize using GNU Parallel: {{{#!highlight bash parallel ssh -- {/} bash "<" {} ::: /etc/myapp/* }}} ==== Associative array hacks in older shells ==== Before you think of using `eval` to mimic associative arrays in an older shell (probably by creating a set of variable names like `homedir_alex`), try to think of a simpler or completely different approach that you could use instead. If this hack still seems to be the best thing to do, consider the following disadvantages: 1. It's really hard to read, to keep track of, and to maintain. 1. The variable names must be a single line and match the RegularExpression {{{^[a-zA-Z_][a-zA-Z_0-9]*$}}} -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames, for instance -- consider a user named {{{hong-hu}}}. A dash '-' cannot be part of a variable name, so the entire attempt to make a variable named `homedir_hong-hu` is doomed from the start. 1. Quoting is hard to get right. If a content string (not a variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it through both shell parsings. And that's just for ''constants'', known at the time you write the program. (Bash's `printf %q` helps, but nothing analogous is available in POSIX shells.) 1. If the program handles unsanitized user input, it can be [[BashFAQ/048|VERY dangerous]]! Read [[BashGuide/Arrays]] or [[BashFAQ/005]] for a more in-depth description and examples of how to use arrays in Bash. If you ''need'' an associative array but your shell doesn't support them, please consider using AWK instead. === Name References === ksh93 introduced ''name references'', which are variables that work like symbolic links. The content of a nameref variable is the name of a second variable. Assignments, expansions, and other operations on a nameref variable are redirected to the variable they "point to". `nameref` variables were subsequently ported to mksh, bash 4.3, and zsh versions > 5.9. Currently, the bash, mksh, and zsh implementations do not support the full power of the ksh93 nameref system. * The zsh implementation is currently the most complete in comparison with ksh93, supporting most features including reference parameters to a limited degree, which have been adapted to support its dynamic scoping system. Variables passed to functions by reference in zsh work correctly in most cases. In certain complex cases the implementation is imperfect and several details aren't quite correct yet. These are probably fixable and could be addressed in a future release. It is also the newest implementation, currently unreleased. * The bash implementation is less complete. reference parameters are completely unsupported as of 5.3, thus the name that's pointed to by a variable can only be resolved using the same dynamic scoping rules as any other variable name. For example, if you have `declare -n ref=$1` and try to use `$ref`, the shell will look for whatever variable of that name is visible by the current function. There is no way to override this. You cannot say "this nameref points to the variable `foo` in the caller's scope" unambiguously. * The mksh implementation shares all of bash's limitations and additionally lacks support for the special `for` loop iteration feature over nameref variables. Namerefs are an mksh extension - oksh and other pdksh derivs have no nameref support. {{{#!highlight bash # ksh93 / Bash >= 4.3 / zsh > 5.9 / mksh (no printf builtin) realvariable=contents typeset -n ref=realvariable printf '%s=%q\n' "${!ref}" "$ref" # print the name and contents of the real variable }}} As long as you avoid namespace collisions, namerefs can be extremely useful. They give the "indirection" that many people are looking for: {{{#!highlight bash arr1=(first array) arr2=(second array) declare -n ref if [[ $someoption ]]; then ref=arr2 else ref=arr1 fi for i in "${ref[@]}"; do ...; done }}} === Indirection === In this section, we discuss various other tricks available in some older shells where namerefs aren't available. ==== Think before using indirection ==== Putting variable names or any other [[BashFAQ/050|bash syntax inside parameters]] is frequently done incorrectly and in inappropriate situations to solve problems that have better solutions. It violates the separation between code and data, and as such puts you on a slippery slope toward bugs and security issues. Indirection can make your code less transparent and harder to follow. Normally, in bash scripting, you won't need indirect references at all. Generally, people look at this for a solution when they don't understand or know about [[BashGuide/Arrays|Bash Arrays]] (indexed or associative) or haven't fully considered other Bash features such as functions. ==== Evaluating indirect/reference variables ==== [[BASH]] allows for expanding parameters ''indirectly'' -- that is, one variable may contain the name of another variable. Name reference variables are the preferred method for performing variable indirection. Older versions of Bash could also use a `!` prefix operator in parameter expansions for variable indirection. Namerefs should be used unless portability to older bash versions is required. No other shell uses `${!variable}` for indirection and there are problems relating to use of that syntax for this purpose. It is also less flexible. {{{#!highlight bash # Bash realvariable=contents ref=realvariable printf '%s\n' "${!ref}" # prints the contents of the real variable }}} Zsh allows you to access a parameter indirectly with the parameter expansion flag `P`: {{{#!highlight bash # zsh realvariable=contents ref=realvariable echo ${(P)ref} # prints the contents of the real variable }}} zsh's ability to nest parameter expansions allow for referencing [[BashFAQ/005|arrays]] too: {{{#!highlight bash # zsh myfunc() { local ref=$1 echo "array $1 has ${#${(@P)ref}} elements" } realarray=(...) myfunc realarray }}} Unfortunately, for shells other than Bash, ksh93, and zsh there is no syntax for ''evaluating'' a referenced variable. You would have to use [[BashFAQ/048|eval]], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe. It's difficult to imagine a practical use for this that wouldn't be just as easily performed by using an associative array. But people ask it all the time (it is genuinely a ''frequently'' asked question). We are not aware of any trick that can duplicate that functionality in POSIX or Bourne shells without `eval`, which can be difficult to do securely. Older versions of Bash can ''almost'' do it -- some indirect array tricks work, and others do not, and we do not know whether the syntax involved will remain stable in future releases. So, consider this a ''use at your own risk'' hack. {{{#!highlight bash # Bash -- trick #1. Works in bash 2 and up, and ksh93v+ (when invoked as bash) realarray=(...) ref=realarray; index=2 tmp=${ref}[index] echo "${!tmp}" # gives array element [2] }}} {{{#!highlight bash # Bash -- trick #2. Seems to work in bash 3 and up. # Can't be combined with special expansions until 4.3. e.g. "${!tmp##*/}" # Does NOT work in bash 2.05b -- Expands to one word instead of three in bash 2. tmp=${ref}[@] printf '<%s> ' "${!tmp}"; echo # Iterate whole array as one word per element. }}} It is not possible to retrieve array indices directly using the Bash ''${!var}'' indirect expansion. ==== Assigning indirect/reference variables ==== Sometimes you'd like to "point" from one variable to another, for purposes of writing information to a dynamically configurable place. Typically this happens when you're trying to write a "reusable" function or library, and you want it to [[BashFAQ/084|put its output]] in a variable of the caller's choice instead of the function's choice. (Various traits of Bash make safe [[BashWeaknesses|reusability of Bash functions]] difficult at best, so this is something that should not happen ''often''.) Assigning a value "through" a reference (I'm going to use "ref" from now on) is more widely possible, but the means of doing so are usually extremely shell-specific. All shells with the sole exception of AT&T ksh93 lack real reference variables or pointers. Indirection can '''only''' be achieved by indirectly evaluating variable names. IOW, you can never have a real unambiguous reference to an object in memory; the best you can do is use the name of a variable to try simulating the effect. Therefore, '''you must control the value of the ref''' and ensure side-effects such as globbing, user-input, and conflicting local parameters can't affect parameter names. Names must either be deterministic or validated in a way that makes certain guarantees. If an end user can populate the ref variable with arbitrary strings, the result can be unexpected code injection. We'll show an example of this at the end. In ksh93, we can use `nameref` again: {{{#!highlight bash # ksh93/mksh/Bash 4.3 typeset -n ref=realvariable ref=contents # realvariable now contains the string "contents" }}} In zsh, using parameter expansions `::=` and expansion flags `P`: {{{#!highlight bash # zsh ref=realvariable : ${(P)ref::=contents} # redefines realvariable unconditionally to the string "contents" }}} In Bash, if you only want to assign '''a single line''' to the variable, you can use `read` and Bash's [[HereDocument#Here_Strings|here string]] syntax: {{{#!highlight bash # Bash/ksh93/mksh/zsh ref=realvariable IFS= read -r -- "$ref" <<<"contents" # realvariable now contains the string "contents" }}} If you need to assign '''multiline values''', you can use a [[HereDocument]]: {{{#!highlight bash # Bash ref=realvariable IFS= read -r -d '' -- "$ref" <&2 "usage: f varname" return 1 fi if printf '%s\n' "$1" | LC_ALL=C grep -v -q '^[A-Za-z_][A-Za-z0-9_]*$' || [ "$(printf '%s\n' "$1" | wc -l)" != 1 ]; then echo >&2 "f: invalid varname argument" return 2 fi # Code goes here that eventually sets the variable "x". # In shells with local variables, x should be local. # x contains the value we'd like to return to the caller. x=foo # Return the value into the caller's variable. eval "${1}=\$x" } }}} === See Also === * [[ https://web.archive.org/web/20230402053523/https://wiki.bash-hackers.org/syntax/arrays#indirection | More advanced indirection on arrays]] * [[https://gist.github.com/ormaaj/5682807 | Bash vs Mksh vs ksh93 namerefs]] * [[BashFAQ/005]] * [[BashGuide/Arrays]] * [[BashSheet#Arrays|BashSheet Array reference]] ---- CategoryShell