Differences between revisions 5 and 6
Revision 5 as of 2008-05-08 23:20:52
Size: 4098
Editor: GreyCat
Comment: clean up
Revision 6 as of 2008-05-30 18:06:25
Size: 5629
Editor: GreyCat
Comment: add ksh93 namerefs, clean up a LOT.
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
Before starting to use dynamically created variables, think again of a simpler approach. If it still seems to be the best thing to do, have a look at the following disadvantages:
Line 5: Line 4:
 1. It's hard to read and to maintain.
 1. The variable names must match the regular expression {{{^[a-zA-Z_][a-zA-Z_0-9]*}}} -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames; consider a user named {{{hong-hu}}}. A dash '-' cannot be a valid part of a variable name.
 1. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it.
 1. If the program handles unsanitized user input, it can be [:BashFAQ/048:VERY dangerous]!

Bash (but not Korn shell, POSIX or Bourne shell) allows you to expand a parameter ''indirectly'' -- that is, one variable may contain the name of another variable:
=== Evaluating indirect/reference variables ===
[:BASH:] allows you to expand a parameter ''indirectly'' -- that is, one variable may contain the name of another variable:
Line 17: Line 12:
This works for evaluating, but not for assigning a value. In order to assign a value "through" a reference (or pointer, or indirect variable, or whatever you want to call it -- I'm going to use "ref" from now on), you have to resort to tricks. KornShell (ksh93) has a completely different, more powerful syntax -- the `nameref` command (also known as `typeset -n`):
  {{{
  # ksh93
  realvariable=contents
  nameref ref=realvariable
  echo "$ref" # prints the contents of the real variable}}}
Line 19: Line 19:
One such trick is to use {{{read}}} and Bash's ''here string'' syntax: ksh93's `nameref` allows us to work with references to arrays, as well as regular scalar variables. For example,
  {{{
  # ksh93
  myfunc() {
    nameref ref=$1
    echo "array $1 has ${#ref[*]} elements"
  }
  realarray=(...)
  myfunc realarray}}}

We are not aware of any trick that can duplicate that functionality in Bash, POSIX or Bourne shells (short of using [:BashFAQ/048:eval], which is extremely difficult to do securely).

Unfortunately, for shells other than Bash and ksh93, there is no syntax for ''evaluating'' a referenced variable. You would have to use [:BashFAQ/048:eval], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.

=== Assigning indirect/reference variables ===
Assigning a value "through" a reference (or pointer, or indirect variable, or whatever you want to call it -- I'm going to use "ref" from now on) is more widely possible, but the means of doing so are extremely shell-specific.

In ksh93, we can just use `nameref` again:
  {{{
  # ksh93
  nameref ref=realvariable
  ref="contents"
  # realvariable now contains the string "contents"}}}

In Bash, we can use {{{read}}} and Bash's ''here string'' syntax:
Line 33: Line 57:
Another is to use Bash's {{{printf -v}}} (only available in [:BashFAQ/061:recent versions]): Another trick is to use Bash's {{{printf -v}}} (only available in [:BashFAQ/061:recent versions]):
Line 41: Line 65:
Yet another is Korn shell's {{{typeset}}} or Bash's {{{declare}}}. These are roughly equivalent to each other. Both of them cause a variable to become ''locally scoped'' to a function, if used inside a function; but if used outside a function, they can substitute for {{{read}}} in this case: Yet another trick is Korn shell's {{{typeset}}} or Bash's {{{declare}}}. These are roughly equivalent to each other. Both of them cause a variable to become ''locally scoped'' to a function, if used inside a function; but if used outside a function, they can operate on global variables.
Line 44: Line 68:
  # Korn shell:   # Korn shell (all versions):
Line 49: Line 73:

The advantage of using `typeset` or `declare` over `eval` is that the right hand side of the assignment is ''not'' parsed by the shell. If you used `eval` here, you would have to sanitize/escape the entire right hand side first.
Line 60: Line 86:
Unfortunately, for shells other than Bash, there is no syntax for ''evaluating'' a referenced variable. You would have to use [:BashFAQ/048:eval], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.
=== Associative Arrays ===
Line 76: Line 101:
BASH (including version 3.x) does not support them, unfortunately. Either use [:BashFAQ/048:eval] after sanitizing your data, or switch to awk, perl, ksh93, tcl, etc. BASH (including version 3.x) does not support them, unfortunately. Consider switching to awk, perl, ksh93, tcl, etc. if you need this type of data structure to solve your problem.

Before you think of using `eval` to mimic this behavior in a shell (probably by creating a set of variable names like `homedir_alex`), try to think of a simpler approach that you could use instead. If this hack still seems to be the best thing to do, have a look at the following disadvantages:

 1. It's hard to read and to maintain.
 1. The variable names must match the RegularExpression {{{^[a-zA-Z_][a-zA-Z_0-9]*}}} -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames, for instance -- consider a user named {{{hong-hu}}}. A dash '-' cannot be part of a variable name, so the entire attempt to make a variable named `homedir_hong-hu` is doomed from the start.
 1. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it through both shell parsings. And that's just for ''constants'', known at the time you write the program.
 1. If the program handles unsanitized user input, it can be [:BashFAQ/048:VERY dangerous]!

Anchor(faq6)

How can I use variable variables (indirect variables, pointers, references) or associative arrays?

Evaluating indirect/reference variables

[:BASH:] allows you to expand a parameter indirectly -- that is, one variable may contain the name of another variable:

  •   # Bash
      realvariable=contents
      ref=realvariable
      echo "${!ref}"   # prints the contents of the real variable

KornShell (ksh93) has a completely different, more powerful syntax -- the nameref command (also known as typeset -n):

  •   # ksh93
      realvariable=contents
      nameref ref=realvariable
      echo "$ref"      # prints the contents of the real variable

ksh93's nameref allows us to work with references to arrays, as well as regular scalar variables. For example,

  •   # ksh93
      myfunc() {
        nameref ref=$1
        echo "array $1 has ${#ref[*]} elements"
      }
      realarray=(...)
      myfunc realarray

We are not aware of any trick that can duplicate that functionality in Bash, POSIX or Bourne shells (short of using [:BashFAQ/048:eval], which is extremely difficult to do securely).

Unfortunately, for shells other than Bash and ksh93, there is no syntax for evaluating a referenced variable. You would have to use [:BashFAQ/048:eval], which means you would have to undergo extreme measures to sanitize your data to avoid catastrophe.

Assigning indirect/reference variables

Assigning a value "through" a reference (or pointer, or indirect variable, or whatever you want to call it -- I'm going to use "ref" from now on) is more widely possible, but the means of doing so are extremely shell-specific.

In ksh93, we can just use nameref again:

  •   # ksh93
      nameref ref=realvariable
      ref="contents"
      # realvariable now contains the string "contents"

In Bash, we can use read and Bash's here string syntax:

  •   # Bash
      ref=realvariable
      read $ref <<< "contents"
      # realvariable now contains the string "contents"

This works equally well with Bash array variables too:

  •   # Bash
      aref=realarray
      read -a $aref <<< "words go into array elements"
      echo "${realarray[1]}"   # prints "go"

Another trick is to use Bash's printf -v (only available in [:BashFAQ/061:recent versions]):

  •   # Bash 3.1 or higher
      ref=realvariable
      printf -v $ref "contents"

The printf -v trick is handy if your contents aren't a constant string, but rather, something dynamically generated. You can use all of printf's formatting capabilities.

Yet another trick is Korn shell's typeset or Bash's declare. These are roughly equivalent to each other. Both of them cause a variable to become locally scoped to a function, if used inside a function; but if used outside a function, they can operate on global variables.

  •   # Korn shell (all versions):
      typeset $ref="contents"
    
      # Bash:
      declare $ref="contents"

The advantage of using typeset or declare over eval is that the right hand side of the assignment is not parsed by the shell. If you used eval here, you would have to sanitize/escape the entire right hand side first.

If you aren't using Bash or Korn shell, you can still do assignments to referenced variables using here document syntax:

  •   # Bourne
      ref=realvariable
      read $ref <<EOF
      contents
      EOF

Remember that, when using a here document, if the sentinel word (EOF in our example) is unquoted, then parameter expansions will be performed inside the body. If the sentinel is quoted, then parameter expansions are not performed. Use whichever is more convenient for your task.

Associative Arrays

Sometimes it's convenient to have associative arrays, arrays indexed by a string. Awk has associative arrays. Perl calls them "hashes", while Tcl simply calls them "arrays". [:KornShell:ksh93] supports this kind of array:

  •  # ksh93
     typeset -A homedir             # Declare ksh93 associative array
     homedir[jim]=/home/jim
     homedir[silvia]=/home/silvia
     homedir[alex]=/home/alex
     
     for user in ${!homedir[@]}     # Enumerate all indices (user names)
     do
         echo "Home directory of user $user is ${homedir[$user]}"
     done

BASH (including version 3.x) does not support them, unfortunately. Consider switching to awk, perl, ksh93, tcl, etc. if you need this type of data structure to solve your problem.

Before you think of using eval to mimic this behavior in a shell (probably by creating a set of variable names like homedir_alex), try to think of a simpler approach that you could use instead. If this hack still seems to be the best thing to do, have a look at the following disadvantages:

  1. It's hard to read and to maintain.
  2. The variable names must match the RegularExpression ^[a-zA-Z_][a-zA-Z_0-9]* -- i.e., a variable name cannot contain arbitrary characters but only letters, digits, and underscores. We cannot have a variable's name contain Unix usernames, for instance -- consider a user named hong-hu. A dash '-' cannot be part of a variable name, so the entire attempt to make a variable named homedir_hong-hu is doomed from the start.

  3. Quoting is hard to get right. If content strings (not variable name) can contain whitespace characters and quotes, it's hard to quote it right to preserve it through both shell parsings. And that's just for constants, known at the time you write the program.

  4. If the program handles unsanitized user input, it can be [:BashFAQ/048:VERY dangerous]!

BashFAQ/006 (last edited 2023-04-14 06:52:11 by ormaaj)