Differences between revisions 3 and 9 (spanning 6 versions)
Revision 3 as of 2007-05-07 17:47:48
Size: 1685
Editor: GreyCat
Comment: expand
Revision 9 as of 2009-01-22 08:56:17
Size: 5127
Editor: Lhunath
Comment: explain a bit more why functions solve the problem
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
[[Anchor(faq50)]]
== I'm trying to construct a command dynamically, but I can't figure out how to deal with quoted multi-word arguments. ==
<<Anchor(faq50)>>
== I'm trying to put a command in a variable, but the complex cases always fail! ==
Line 11: Line 10:
This fails because of word-splitting. When {{{$args}}} is evaluated, it becomes four words: {{{'The}}} is the second word, and {{{subject'}}} is the third word. This fails because of WordSplitting. When {{{$args}}} is expanded, it becomes four words. {{{'The}}} is the second word, and {{{subject'}}} is the third word.
Line 13: Line 12:
What's needed is a way to maintain each word as a separate item, even if that word contains multiple spaces. Quotes won't do it, but an [#faq5 array] will. The solution to the problem is in 90% of the cases a completely different approach. You generally do NOT want to put command names or command options in variables. Variables should contain the data you are trying to pass to the command, like usernames, hostnames, ports, text, etc. They should NOT contain options that are specific to one certain command. Those things belong in '''functions'''. What you probably should be doing, is this:

{{{
    # Send an email to someone.
    # Sends an email to the address specified as the first argument,
    # optionally with the subject specified by the second.
    # Reads the body of the mail from standard input.
    #
    # sendto address [subject]
    #
    sendto() {
        mail ${2:+-s "$2"} "$1"
    }

    sendto "$address" "The Subject" < "$bodyfile"
}}}

Most often, this problem arises because somebody wants to an option to a command only if it's necessary. In our example, somebody might want to only specify a subject to the message if he has one, and think this is the way to do it:
{{{
    if [[ $subject ]]; then
        args+="-s $subject"
    fi
}}}

Unfortunately, as explained above, this does not work. It is not permitting of white space (which we all know occurs plenty in subject strings). The function provides a solution to this, because now, we don't need to dynamically construct the arguments; we just do this:

{{{
    sendto "$address" "$subject"
}}}

And the function checks if `$subject` has expanded to anything. If it has, the function adds the `-s "$subject"` to the `mail` command. If it hasn't, the function doesn't add the `-s` option at all.

Unless you '''KNOW''' that this is not what you want, stop reading now.

The root of the issue described above is the fact that you need a way to maintain each argument as a separate word, even if that argument contains spaces. Quotes won't do it, but an [[BashFAQ/005|array]] will.

As such, if you really wanted to create a command dynamically; put each argument in a separate element of an array, like so:
Line 21: Line 56:
Often, this question arises when someone is trying to use {{{dialog}}} to construct a menu on the fly. For an example of how to do this properly, see [#faq40 FAQ #40]. Often, this question arises when someone is trying to use {{{dialog}}} to construct a menu on the fly. The {{{dialog}}} command can't be hard-coded, because its parameters are supplied based on data only available at run time. For an example of how to do this properly, see [[BashFAQ/040|FAQ #40]].
Line 23: Line 58:
Another reason people attempt to do this is because they want to {{{echo "I am going to run this command: $command"}}} before they actually run it. If that's all you want, then simply use the {{{set -x}}} command, or invoke your script with {{{#!/bin/bash -x}}} or {{{bash -x ./myscript}}}. Note that you can turn it off and back on inside the script with {{{set +x}}} and {{{set -x}}}. Another reason people attempt to do this is because they want their script to print each command before it runs it. If that's all you want, then simply use the {{{set -x}}} command, or invoke your script with {{{#!/bin/bash -x}}} or {{{bash -x ./myscript}}}. Note that you can turn it off and back on inside the script with {{{set +x}}} and {{{set -x}}}.
Line 25: Line 60:
It's worth noting that you ''cannot'' put a pipeline command into an array variable and then execute it using the {{{"${array[@]}"}}} technique. The only way to store a pipeline in a variable would be to add (carefully!) a layer of quotes if necessary, store it in a string variable, and then use {{{eval}}} or {{{sh}}} to run the variable. This is [#faq48 not recommended], for security reasons. It's worth noting that you ''cannot'' put a pipeline command into an array variable and then execute it using the {{{"${array[@]}"}}} technique. The only way to store a pipeline in a variable would be to add (carefully!) a layer of quotes if necessary, store it in a string variable, and then use {{{eval}}} or {{{sh}}} to run the variable. This is [[BashFAQ/048|not recommended]], for security reasons. The same thing applies to commands involving redirection, {{{if}}} or {{{while}}} statements, and so on.

Some people get into trouble because they want to have their script print their commands ''including redirections'' before it runs them. {{{set -x}}} shows the command without redirections. People try to work around this by doing things like:
{{{
    # Non-working example
    command="mysql -u me -p somedbname < file"
    ((DEBUG)) && echo "$command"
    "$command"
}}}
(This is so common that I include it explicitly, even though it's repeating what I already wrote.)

Once again, ''this does not work''. Not even using an array works here. The only thing that would work is rigorously escaping the command to be sure ''no'' metacharacters will cause serious security problems, and then using `eval` or `sh` to re-read the command. '''Please don't do that!'''

If your head is SO far up your ass that you still think you need to write out every command you're about to run before you run it, AND that you must include all redirections, then just do this:
{{{
    # Working example
    echo "mysql -u me -p somedbname < file"
    mysql -u me -p somedbname < file
}}}
Don't use a variable at all. Just copy and paste the command, wrap an extra layer of quotes around it (sometimes tricky), and stick an `echo` in front of it.

My personal recommendation would be just to use {{{set -x}}} and not worry about it.

I'm trying to put a command in a variable, but the complex cases always fail!

Some people attempt to do things like this:

    # Non-working example
    args="-s 'The subject' $address"
    mail $args < $body

This fails because of WordSplitting. When $args is expanded, it becomes four words. 'The is the second word, and subject' is the third word.

The solution to the problem is in 90% of the cases a completely different approach. You generally do NOT want to put command names or command options in variables. Variables should contain the data you are trying to pass to the command, like usernames, hostnames, ports, text, etc. They should NOT contain options that are specific to one certain command. Those things belong in functions. What you probably should be doing, is this:

    # Send an email to someone.
    # Sends an email to the address specified as the first argument,
    # optionally with the subject specified by the second.
    # Reads the body of the mail from standard input.
    #
    # sendto address [subject]
    #
    sendto() {
        mail ${2:+-s "$2"} "$1"
    }

    sendto "$address" "The Subject" < "$bodyfile"

Most often, this problem arises because somebody wants to an option to a command only if it's necessary. In our example, somebody might want to only specify a subject to the message if he has one, and think this is the way to do it:

    if [[ $subject ]]; then
        args+="-s $subject"
    fi

Unfortunately, as explained above, this does not work. It is not permitting of white space (which we all know occurs plenty in subject strings). The function provides a solution to this, because now, we don't need to dynamically construct the arguments; we just do this:

    sendto "$address" "$subject"

And the function checks if $subject has expanded to anything. If it has, the function adds the -s "$subject" to the mail command. If it hasn't, the function doesn't add the -s option at all.

Unless you KNOW that this is not what you want, stop reading now.

The root of the issue described above is the fact that you need a way to maintain each argument as a separate word, even if that argument contains spaces. Quotes won't do it, but an array will.

As such, if you really wanted to create a command dynamically; put each argument in a separate element of an array, like so:

    # Working example
    args=(-s "The subject" "$address")
    mail "${args[@]}" < $body

Often, this question arises when someone is trying to use dialog to construct a menu on the fly. The dialog command can't be hard-coded, because its parameters are supplied based on data only available at run time. For an example of how to do this properly, see FAQ #40.

Another reason people attempt to do this is because they want their script to print each command before it runs it. If that's all you want, then simply use the set -x command, or invoke your script with #!/bin/bash -x or bash -x ./myscript. Note that you can turn it off and back on inside the script with set +x and set -x.

It's worth noting that you cannot put a pipeline command into an array variable and then execute it using the "${array[@]}" technique. The only way to store a pipeline in a variable would be to add (carefully!) a layer of quotes if necessary, store it in a string variable, and then use eval or sh to run the variable. This is not recommended, for security reasons. The same thing applies to commands involving redirection, if or while statements, and so on.

Some people get into trouble because they want to have their script print their commands including redirections before it runs them. set -x shows the command without redirections. People try to work around this by doing things like:

    # Non-working example
    command="mysql -u me -p somedbname < file"
    ((DEBUG)) && echo "$command"
    "$command"

(This is so common that I include it explicitly, even though it's repeating what I already wrote.)

Once again, this does not work. Not even using an array works here. The only thing that would work is rigorously escaping the command to be sure no metacharacters will cause serious security problems, and then using eval or sh to re-read the command. Please don't do that!

If your head is SO far up your ass that you still think you need to write out every command you're about to run before you run it, AND that you must include all redirections, then just do this:

    # Working example
    echo "mysql -u me -p somedbname < file"
    mysql -u me -p somedbname < file

Don't use a variable at all. Just copy and paste the command, wrap an extra layer of quotes around it (sometimes tricky), and stick an echo in front of it.

My personal recommendation would be just to use set -x and not worry about it.

BashFAQ/050 (last edited 2024-04-15 23:48:57 by larryv)