Diff for "BashFAQ/092"

How do I write a CGI script that accepts parameters?

There are always circumstances beyond our control that drive us to do things that we would never choose to do on our own. This FAQ entry describes one of those situations.

A CGI program can be invoked with parameters, sent by the web browser (user agent). There are (at least) two ways to invoke a CGI program: the "GET" method and the "POST" method. In the "GET" method, parameters are provided to the CGI program in an environment variable called QUERY_STRING. The parameters take the form of KEY=VALUE definitions (e.g. user=george), with some characters encoded in hexadecimal, spaces encoded as plus signs, all joined together with ampersands. In the "POST" method, the parameters are provided on standard input instead.

Now of course we know you would never write a CGI script in Bash. So for the purposes of this entry we will assume that terrorists have kidnapped your spouse and children and will torture, maim, kill, "or worse" them if you do not comply with their demands to write such a script.

(The "or worse" situation would clearly be something like being forced to use Microsoft based software.)

So, given a QUERY_STRING variable, we would like to extract the keys (variables) and their values, so that we can use them in the script.

The quick, easy and dangerous way to process the QUERY_STRING is to convert the &s to ;s and then use the eval command to run those assignments. However, the use of eval is STRONGLY DISCOURAGED. That is to say we always avoid using eval if there is any way around it.

The Dangerous Way

# Read in the cgi input string
if [ "$QUERY_STRING" ]; then
  foo=$QUERY_STRING
else
  read foo
fi

# Convert some of the encoded strings and things like "&" (left as an exercise for the reader)

# Run eval on the string
eval $foo

# Sit back and discover that the user has put "/bin/rm -rf /" in one of the web form fields,
# which even if not root will do damage to some part of the file system.
# Another dangerous string would be a fork bomb.

A Safer Way

Instead of telling the shell to execute whatever code the user provided in the parameters, a better approach is to extract each variable/value pair, and assign them to shell variables, one by one, without executing them. This requires an indirect variable assignment, which means using some shell-specific trickery. We'll write this using Bash syntax; converting to ksh or Bourne shell is left as an exercise.

# Bash

# Read in the cgi input string
if [ "$QUERY_STRING" ]; then
  foo=$QUERY_STRING
else
  read -r foo
fi

# foo contains something like name=Fred+Flintstone&city=Bedrock
# Treat this as a list of key=value expressions joined with &.
# Iterate through the list and perform each assignment.

IFS='&'; set -f
for i in $foo; do
    declare "$i"
done
unset IFS

# Each CGI parameter will now be in a shell variable of the same name.
# You'd better know what the names are, because we didn't keep track.
# Each variable is still "urlencoded".  Spaces are encoded as + and
# various things are encoded as %xx where xx is hexadecimal.

# Suppose we want to use a parameter named "name".

# First, decode the spaces.

name=${name//+/ }

# Now decode the %xx characters.  We use another trick to do this.
# First, we replace all % signs with \x
# Second, we use echo -e to cause all the \xxx to be evaluated.

name=${name//\%/\\x}
name=$(echo -e "$name")

# We did not do this BEFORE the iteration/assignment loop because if we had,
# then a parameter that contains an encoded & (or whatever malicious character)
# would have caused much grief.  We have to do it here.

# Now you do whatever you wanted to do with "name".

While this might be a little less clear, it avoids this huge security problem that eval has: executing any arbitrary command the user might care to enter into the web form. Clearly this is an improvement.

There are still some imperfections in this version. For example, we do not perform any validation on the left hand side (the variable name) in each key=value pair to ensure that it's a valid, or safe, shell variable name. What if the user passes PATH= in a query parameter?

Associative Arrays

An even better approach might be to place the key/value pairs into an associative array. Associative arrays are available in ksh93 and in bash 4.0, but not in POSIX or Bourne shells. They are designed to hold key/value pairs where the keys can be arbitrary strings, so they seem appropriate for this job.

# Bash 4+

# Read in the cgi input string
if [ "$QUERY_STRING" ]; then
  foo=$QUERY_STRING
else
  read -r foo
fi

# Set up an associative array to hold the query parameters.
declare -A q

# Iterate through the key=value+%41%42%43 elements.
# Separate key and value, and perform decoding on the value.
while IFS='=' read -r -d '&' key value <<< "$i&"

    # Decoding steps: first, sanitize -- remove all backslashes.
    # Second, plus signs become spaces.
    # Third, percent signs become \x.
    # This leaves nothing that can unexpectedly trigger a printf expansion.
    # All backslashes are ours, and no percent signs remain.

    value=${value//\\/}
    value=${value//+/ }
    value=${value//\%/\\x}
    printf -v final -- "$value"
    q["$key"]="$final"
done

# Now we can use the parameters from the associative array named q.
# If we need a list of the keys, it's ${!q[*]}.

The sanitization step is extremely important here. Without that precaution, the printf might be vulnerable to a format string attack. The printf -v varname option is available in every version of bash that supports associative arrays, so we may use it here. It's much more efficient than calling a SubShell. We've also avoided the potential problems with echo -e if the value happens to be something like -n.

Technically, the CGI specification allows multiple instances of the same key in a single query. For example, group=managers&member=Alice&member=Charlie is a perfectly legitimate query string. None of the approaches on this page handle this case (at least not in what we'd probably consider the "correct" way). Fortunately, it's not often that you'd write a CGI like this; and in any case, you're not being forced to use bash for this task.