Differences between revisions 2 and 3
Revision 2 as of 2007-05-07 19:15:30
Size: 3688
Editor: GreyCat
Comment: Why didn't this already link to #48?
Revision 3 as of 2007-11-08 14:52:05
Size: 4066
Editor: GreyCat
Comment: remove all eval examples. give better examples.
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
== How can I use associative arrays or variable variables? == == How can I use variable variables (indirect variables, pointers, references) or associative arrays? ==
Before starting to use dynamically created variables, think again of a simpler approach. If it still seems to be the best thing to do, have a look at the following disadvantages:
Line 4: Line 5:
Sometimes it's convenient to have associative arrays, arrays indexed by a string. Perl calls them "hashes", while Tcl simply calls them "arrays". KornShell93 already supports this kind of array:  1. It's hard to read and to maintain.
 1. The variable names must match the regular expression {{{^[a-zA-Z_][a-zA-Z_0-9]*}}} -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames; consider a user named {{{hong-hu}}}. A dash '-' cannot be a valid part of a variable name.
 1. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it.
 1. If the program handles unsanitized user input, it can be [#faq48 VERY dangerous]!

Bash (but not Korn shell, POSIX or Bourne shell) allows you to expand a parameter ''indirectly'' -- that is, one variable may contain the name of another variable:
  {{{
  realvariable=contents
  ref=realvariable
  echo "${!ref}" # prints the contents of the real variable}}}

This works for evaluating, but not for assigning a value. In order to assign a value "through" a reference (or pointer, or indirect variable, or whatever you want to call it -- I'm going to use "ref" from now on), you have to resort to tricks.

One such trick is to use {{{read}}} and Bash's ''here string'' syntax:
  {{{
  ref=realvariable
  read $ref <<< "contents"
  # realvariable now contains the string "contents"}}}

This works equally well with Bash array variables too:
  {{{
  aref=realarray
  read -a $aref <<< "words go into array elements"
  echo "${realarray[1]}" # prints "go"}}}

Another is to use Bash's {{{printf -v}}} (only available in [#faq61 recent versions]):
  {{{
  ref=realvariable
  printf -v $ref "contents"}}}

The {{{printf -v}}} trick is handy if your contents aren't a constant string, but rather, something dynamically generated. You can use all of {{{printf}}}'s formatting capabilities.

Yet another is Korn shell's {{{typeset}}} or Bash's {{{declare}}}. These are roughly equivalent to each other. Both of them cause a variable to become ''locally scoped'' to a function, if used inside a function; but if used outside a function, they can substitute for {{{read}}} in this case:

  {{{
  # Korn shell:
  typeset $ref="contents"

  # Bash:
  declare $ref="contents"}}}

If you aren't using Bash or Korn shell, you can still do assignments to referenced variables using ''here document'' syntax:
  {{{
  # Portable code.
  ref=realvariable
  read $ref <<EOF
  contents
  EOF}}}

Remember that, when using a here document, if the sentinel word ({{{EOF}}} in our example) is unquoted, then parameter expansions will be performed inside the body. If the sentinel is quoted, then parameter expansions are not performed. Use whichever is more convenient for your task.

Unfortunately, for shells other than Bash, there is no syntax for ''evaluating'' a referenced variable. You would have to use [#faq48 eval], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.

Sometimes it's convenient to have associative arrays, arrays indexed by a string. Awk has associative arrays. Perl calls them "hashes", while Tcl simply calls them "arrays". KornShell93 supports this kind of array:
Line 18: Line 72:
BASH (including version 3.x) does not (yet) support them. However, we could simulate this kind of array by dynamically creating variables like in the following example:

 {{{
 for user in jim silvia alex
 do
     eval homedir_$user=/home/$user
 done}}}

This creates the variables

 {{{
 homedir_jim=/home/jim
 homedir_silvia=/home/silvia
 homedir_alex=/home/alex}}}

with the corresponding content. Note the use of the {{{eval}}} command, which interprets a command line not just one time like the shell usually does, but '''twice'''. In the first step, the shell uses the input {{{homedir_$user=/home/$user}}} to create a new line {{{homedir_jim=/home/jim}}}. In the second step, caused by {{{eval}}}, this variable assignment is executed, actually creating the variable.

Print the variables using

 {{{
 for user in jim silvia alex
 do
     varname=homedir_$user # e.g. "homedir_jim"
     eval varcontent='$'$varname # e.g. "/home/jim"
     echo "home directory of $user is $varcontent"
 done}}}

The {{{eval}}} line needs some explanation. In a first step the command substitution is run:

 {{{
 eval varcontent='$'$varname}}}

becomes

 {{{
 eval varcontent=$homedir_jim}}}

In a second step the {{{eval}}} re-evaluates the line, and converts this to

 {{{
 varcontent=/home/jim}}}

Before starting to use dynamically created variables, think again of a simpler approach. If it still seems to be the best thing to do, have a look at the following disadvantages:

 1. It's hard to read and to maintain.
 1. The variable names must match the regular expression {{{^[a-zA-Z_][a-zA-Z_0-9]*}}} -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. In the example above we could not have processed the home directory of a user named {{{hong-hu}}}, because a dash '-' cannot be a valid part of a variable name.
 1. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it.
 1. If the program handles unsanitized user input, it can be [#faq48 VERY dangerous]!

Here is the summary. "{{{var}}}" is a constant prefix, "{{{$index}}}" contains index string, "{{{$content}}}" is the string to store. Note that quoting is absolutely essential here. A missing backslash \ or a wrong type of quote (e.g. apostrophes '...' instead of quotation marks "...") can (and probably will) cause the examples to fail:

 * Set variables

  {{{
  eval "var$index=\"$content\"" # index must only contain characters from [a-zA-Z0-9_]}}}

 * Print variable content

  {{{
  eval "echo \"var$index=\$$varname\""}}}

 * Check if a variable is empty

  {{{
  if eval "[ -z "\$var$index\" ]"
  then echo "variable is empty: $var$index"
  fi}}}

You've seen the examples. Now maybe you can go a step back and consider using AWK associative arrays, or a multi-line environment variable instead of dynamically created variables.
BASH (including version 3.x) does not support them, unfortunately. Either use [#faq48 eval] after sanitizing your data, or switch to awk, perl, ksh93, tcl, etc.

Anchor(faq6)

How can I use variable variables (indirect variables, pointers, references) or associative arrays?

Before starting to use dynamically created variables, think again of a simpler approach. If it still seems to be the best thing to do, have a look at the following disadvantages:

  1. It's hard to read and to maintain.
  2. The variable names must match the regular expression ^[a-zA-Z_][a-zA-Z_0-9]* -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames; consider a user named hong-hu. A dash '-' cannot be a valid part of a variable name.

  3. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it.
  4. If the program handles unsanitized user input, it can be [#faq48 VERY dangerous]!

Bash (but not Korn shell, POSIX or Bourne shell) allows you to expand a parameter indirectly -- that is, one variable may contain the name of another variable:

  •   realvariable=contents
      ref=realvariable
      echo "${!ref}"   # prints the contents of the real variable

This works for evaluating, but not for assigning a value. In order to assign a value "through" a reference (or pointer, or indirect variable, or whatever you want to call it -- I'm going to use "ref" from now on), you have to resort to tricks.

One such trick is to use read and Bash's here string syntax:

  •   ref=realvariable
      read $ref <<< "contents"
      # realvariable now contains the string "contents"

This works equally well with Bash array variables too:

  •   aref=realarray
      read -a $aref <<< "words go into array elements"
      echo "${realarray[1]}"   # prints "go"

Another is to use Bash's printf -v (only available in [#faq61 recent versions]):

  •   ref=realvariable
      printf -v $ref "contents"

The printf -v trick is handy if your contents aren't a constant string, but rather, something dynamically generated. You can use all of printf's formatting capabilities.

Yet another is Korn shell's typeset or Bash's declare. These are roughly equivalent to each other. Both of them cause a variable to become locally scoped to a function, if used inside a function; but if used outside a function, they can substitute for read in this case:

  •   # Korn shell:
      typeset $ref="contents"
    
      # Bash:
      declare $ref="contents"

If you aren't using Bash or Korn shell, you can still do assignments to referenced variables using here document syntax:

  •   # Portable code.
      ref=realvariable
      read $ref <<EOF
      contents
      EOF

Remember that, when using a here document, if the sentinel word (EOF in our example) is unquoted, then parameter expansions will be performed inside the body. If the sentinel is quoted, then parameter expansions are not performed. Use whichever is more convenient for your task.

Unfortunately, for shells other than Bash, there is no syntax for evaluating a referenced variable. You would have to use [#faq48 eval], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.

Sometimes it's convenient to have associative arrays, arrays indexed by a string. Awk has associative arrays. Perl calls them "hashes", while Tcl simply calls them "arrays". KornShell93 supports this kind of array:

  •  # KornShell93 script - does not work with BASH
     typeset -A homedir             # Declare KornShell93 associative array
     homedir[jim]=/home/jim
     homedir[silvia]=/home/silvia
     homedir[alex]=/home/alex
     
     for user in ${!homedir[@]}     # Enumerate all indices (user names)
     do
         echo "Home directory of user $user is ${homedir[$user]}"
     done

BASH (including version 3.x) does not support them, unfortunately. Either use [#faq48 eval] after sanitizing your data, or switch to awk, perl, ksh93, tcl, etc.

BashFAQ/006 (last edited 2023-04-14 06:52:11 by ormaaj)