## page was renamed from ShellTemplate
= Shell Template Files =

We get lots of questions about how to use a ''template file'', meaning a file with placeholders in it, which we want to replace with values at run time.  There are a few different forms of this question, and as you would expect, a few different answers depending on the exact requirements.

The most basic requirement is what GNU autoconf uses: a pre-made template file with `%PLACEHOLDER%` things in it, and values in specific shell variables that we want to use.  Sometimes, `sed` is OK for this particular task:

{{{
sed -e "s/%PLACEHOLDER%/$variable/g" \
    -e "s/%FIRSTNAME%/$firstname/g" \
    ... "$templatefile" > "$outputfile"
}}}

The notation `%FOO%` is what autoconf actually uses for its placeholders; hence its choice here.

If the variables can contain '''slashes or newlines''' in their contents, then appropriate steps must be taken.  A different delimiter than slash can be used for the `sed` `s///` command, but for newlines, you're advised to consult an [[http://sed.sourceforge.net/sedfaq4.html#s4.1|advanced sed FAQ]].  Fortunately, the case where people want to include newlines in their substitutions, in this form of the question, is somewhat uncommon.

Generally speaking, this usage introduces a [[BashProgramming/05|code injection vulnerability]], in the sense that the contents of your variables are being parsed by `sed` and interpreted as `sed` commands.  The slash and newline special cases are just ''one'' place where the use of `sed` backfires.

If you don't want to work around `sed`'s limitations, or if you're rightfully unwilling to introduce potential code injection vulnerabilities, consider using perl instead:

{{{
PLACEHOLDER=$variable FIRSTNAME=$firstname \
perl -p -e 's/%PLACEHOLDER%/$ENV{"PLACEHOLDER"}/g;
    s/%FIRSTNAME%/$ENV{"FIRSTNAME"}/g;' \
    "$templatefile" > "$outputfile"
}}}

This is completely superior to the `sed` solution in every way except that `sed` is a standard POSIX command, guaranteed to be available, while perl is not.  The perl solution doesn't break with newlines or slashes in the variable contents, and there is no possible code injection.

An alternative form of this question is, "I have a file with shell parameter expansions in it, like `$foo`.  I don't know in advance what variables will be used, so I can't just make a list of all of them in a giant `sed` command."  (Newlines in the variables are much more common here, but fortunately they do not present a problem with the solutions we're about to give.)

If you have GNU gettext installed on your system, you can use a program called `envsubst` to perform the template file substitutions:

{{{
export VAR1 VAR2 ...
envsubst < "$templatefile" > "$outputfile"
}}}

If you don't have `envsubst` but you have Bash, you can use an approach like this:

{{{#!highlight bash
# Bash 3.2 or higher
LC_COLLATE=C
while read -r; do
  while [[ $REPLY =~ \$(([a-zA-Z_][a-zA-Z_0-9]*)|\{([a-zA-Z_][a-zA-Z_0-9]*)\})(.*) ]]; do
    if [[ -z ${BASH_REMATCH[3]} ]]; then   # found $var
      printf %s "${REPLY%"$BASH_REMATCH"}${!BASH_REMATCH[2]}"
    else # found ${var}
      printf %s "${REPLY%"$BASH_REMATCH"}${!BASH_REMATCH[3]}"
    fi
    REPLY=${BASH_REMATCH[4]}
  done
  printf "%s\n" "$REPLY"
done < "$templatefile" > "$outputfile"
}}}

The good:

 * It works without exporting your variables.
 * It doesn't suffer code injection exploits with `$( )` or {{{` `}}}, and EOF in the template won't break it.

The bad:

 * There are some name collision issues: trying to replace `$REPLY` or `$BASH_REMATCH` will use their ''current'' value in the middle of the loop (but it won't loop forever on malicious input). The name collision can be worked around with a minor modification requiring a prefix for each variable:
 {{{#!highlight bash
while read -r; do
  while [[ $REPLY =~ \$(([a-zA-Z_][a-zA-Z_0-9]*)|\{([a-zA-Z_][a-zA-Z_0-9]*)\})(.*) ]]; do
    var=X_${BASH_REMATCH[2]:-${BASH_REMATCH[3]}}
    printf %s "${REPLY%"${BASH_REMATCH[0]}"}${!var}"
    REPLY=${BASH_REMATCH[4]}
  done
  printf "%s\n" "$REPLY"
done < "$templatefile" > "$outputfile"
}}}
 In this case, a `${REPLY}` in the template file will be replaced by the content of the variable `X_REPLY` instead of `REPLY`.

If you need to do it in sh, a portable but potentially dangerous alternative involves constructing a HereDocument and feeding the result to a shell for expansion:

{{{
{ echo "cat <<EOF"
  cat "$templatefile"
  echo "EOF"
} | sh > "$outputfile"
}}}

NOTICE: If you get an unexpected result that includes EOF at the end of the output, your template file probably lacks a newline char at the end.

Recall that substitutions occur in the body of a here document as long as the sentinel (`EOF` in our case) is not quoted on the first line.  We take advantage of that here.

Also, note that the substitutions are performed in a new `sh` shell (or `bash` or `ksh`, whatever you require), not in the current shell.  Therefore, any variables you want to use must be exported.

Since any [[CommandSubstitution|command substitutions]] in the template file will be executed as shell code, it is important that any template file used in this way be under ''your'' control, and not generated by a user.  You've been warned.

----
CategoryShell