Differences between revisions 24 and 25
Revision 24 as of 2014-02-19 21:55:18
Size: 6542
Editor: 109
Comment: [minor] igli: reduce comment
Revision 25 as of 2015-12-21 16:45:58
Size: 5285
Editor: GreyCat
Comment: Add a useful answer, I hope. Remove much cruft. Add a note for historical awks.
Deletions are marked like this. Additions are marked like this.
Line 14: Line 14:
Line 16: Line 17:
=== Use your $HOME ===
Line 17: Line 19:

If you're implementing a daemon which runs under a user account with no home directory, why not simply ''make a private directory for your daemon'' at the same time you're installing the code?
Line 20: Line 24:
=== Making a temporary directory === === Make a temporary directory ===
If you can't use $HOME, the next best answer is to create a private '''directory''' to hold your temp file(s), instead of creating the files directly inside a world-writable sandbox like `/tmp` or `/var/tmp`. The `mktemp` command is atomic, and only reports success if it actually created the directory. So long as we ''do not'' use the `-p` option, we can be assured that it actually created a brand new directory, rather than following a symlink to danger.

Here is one example of this approach:

{{{
# Bash

i=0 tempdir=
trap '[[ $tempdir ]] && rm -rf "$tempdir"' EXIT

while ((++i <= 10)); do
  tempdir=${TMPDIR:-/tmp}/$RANDOM-$$
  mkdir -m 700 "$tempdir" 2>/dev/null && break
done

if ((i > 10)); then
  echo "Could not create temporary directory" >&2
  exit 1
fi
}}}

Instead of `RANDOM`, `awk` can be used to generate a random number in a POSIX compatible way:

{{{
# POSIX

i=0 tempdir=
trap '[ "$tempdir" ] && rm -rf "$tempdir"' EXIT

while [ $i -lt 10 ]; do
  tempdir=${TMPDIR:-/tmp}/$(awk 'BEGIN { srand (); print rand() }')-$$
  mkdir -m 700 "$temp_dir" 2>/dev/null && break
  sleep 1
  i=$((i+1))
done

if [ $i -ge 10 ]; then
  echo "Could not create temporary directory" >&2
  exit 1
fi
}}}

Note however that `srand()` seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack. (Historical awk implementations that predate POSIX may not even use the time of day for `srand()`, so don't count on this if you're on an ancient system.)

Some systems have a 14-character filename limit, so avoid the temptation to string `$RANDOM` together more than twice. You're relying on the atomicity of `mkdir` for your security, ''not'' the obscurity of your random name. If someone fills up `/tmp` with hundreds of thousands of random-number files to thwart you, you've got bigger issues.
Line 27: Line 77:
   # POSIX # Linux
Line 29: Line 79:
   # Set $TMPDIR to "/tmp" only if it didn't have a value previously
   : ${TMPDIR:=/tmp}

   # We can find about $TMPDIR in http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html

   # Create a private temporary directory inside $TMPDIR
   # Remove the temporary directory when the script finishes
   unset temporary_dir
   trap '[ -n "$temporary_dir" ] && rm -rf "$temporary_dir"' EXIT

   save_mask=$(umask)
   umask 077
   temporary_dir=$(mktemp -d "$TMPDIR/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX") || { echo "ERROR creating a temporary file" >&2; exit 1; }
   umask "$save_mask"
unset tempdir
trap '[ "$tempdir" ] && rm -rf "$tempdir"' EXIT
tempdir=$(mktemp -d "${TMPDIR:-/tmp}/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX") ||
  { echo "ERROR creating a temporary file" >&2; exit 1; }
Line 45: Line 85:
And then you can create your particular files inside the temporary folder. And then you can create your particular files inside the temporary directory.
Line 47: Line 87:
----
A different suggestion which does not require a `mktemp` command: a temporary directory can be created that is unlikely to match an existing directory using the {{{RANDOM}}} variable as follows:

{{{
   temp_dir=/tmp/$RANDOM
   mkdir "$temp_dir"
}}}
This will make a directory of the form: {{{/tmp/20445/}}}. To decrease the chance of collision with an existing directory, the {{{RANDOM}}} variable can be used a number of times, or combined with the process ID:

{{{
   temp_dir=/tmp/$RANDOM-$$
   mkdir -m 700 "$temp_dir" || { echo "failed to create temp_dir" >&2; exit 1; }
}}}

This will make a directory of the form {{{/tmp/24953-2875/}}}. This avoids a race condition because the `mkdir` is atomic, as we see in [[BashFAQ/045|FAQ #45]]. It is imperative that you check the result of `mkdir`; if it fails to create the directory, obviously you cannot proceed.

(Some systems have a 14-character filename limit, so avoid the temptation to string `$RANDOM` together more than twice. You're relying on the atomicity of `mkdir` for your security, ''not'' the obscurity of your random name. If someone fills up `/tmp` with hundreds of thousands of random-number files to thwart you, you've got bigger issues.)

----
Instead of {{{RANDOM}}}, `awk` can be used to generate a random number in a POSIX compatible way:

{{{
   temp_dir=/tmp/$(awk 'BEGIN { srand (); print rand() }')
   mkdir -m 700 "$temp_dir"
}}}

Note, however that "srand()" seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack.

Combining the above we can use:
{{{
   # POSIX
   temp_dir=${TMPDIR:=/tmp}/$(awk -v p=$$ 'BEGIN { srand(); s = rand(); sub(/^0./, "", s); printf("%X_%X", p, s) }')
   trap '[ -n "$temp_dir" ] && rm -fr "$temp_dir"' EXIT
   mkdir -m 700 "$temp_dir" || { echo '!! unable to create a tempdir' >&2; temp_dir=; exit 1; }
}}}
Since the default awk numeric format is %g which equates to 6 digits after the decimal point, we have a 6 digit trail. 999,999 is F423F which gives us one character back, another for the {{{_}}} and 8 available for a prefixed pid on a 14-char filesystem. An unsigned 32-bit pid will be a maximum of 8 hex digits, so the above should stand a chance in most places one might encounter such an fs. And of course we bail out as greycat mentioned when the creation fails.

 . ''Isn't this getting just a bit silly?'' -- GreyCat

 . ''No more so than the unportable approach given above. It answers your repeated point about a 14-char limit on filenames, and allows for use of /tmp, which some admins prefer, and can be useful if your script is not a defined program with a specific name, where I agree the better approach is to have a defined directory, under $HOME or elsewhere. As a minor point, the pid prefix enables a degree of control over/ knowledge of any tempdirs created. Personally I'd rather have this, portable to POSIX sh, than a mktemp invocation which I can't rely on.'' -- igli
=== Use platform-specific tools ===
If you're writing a script for only a specific set of systems, and if those systems have a `mktemp` or `tempfile` tool, you can use that. The options will vary from system to system, so you will have to write your script with that in mind. Since some platforms have ''none'' of these tools, this is not a portable solution.
Line 89: Line 91:

I reiterate that the best solution is to use your `$HOME` directory to hold your private files. If you're implementing a daemon or something, which runs under a user account with no home directory, why not simply ''make a private directory for your daemon'' at the same time you're installing the code?

----

How do I create a temporary file in a secure manner?

There does not appear to be any single command that simply works everywhere. tempfile is not portable. mktemp exists more widely (but still not ubiquitously), but it may require a -c switch to create the file in advance; or it may create the file by default and barf if -c is supplied. Some systems don't have either command (Solaris, POSIX).

The traditional answer has usually been something like this:

   # Do not use!  Race condition!
   tempfile=/tmp/myname.$$
   trap 'rm -f "$tempfile"; exit 1' 1 2 3 15
   rm -f "$tempfile"
   touch "$tempfile"

The problem with this is: if the file already exists (for example, as a symlink to /etc/passwd), then the script may write things in places they should not be written. Even if you remove the file immediately before using it, you still have a RaceCondition: someone could re-create a malicious symlink in the interval between your shell commands.

Use your $HOME

The best portable answer is to put your temporary files in your home directory (or some other private directory) where nobody else has write access. Then at least you don't have to worry about malicious users. Simplistic PID-based schemes (or hostname + PID for shared file systems) should be enough to prevent conflicts with your own scripts.

If you're implementing a daemon which runs under a user account with no home directory, why not simply make a private directory for your daemon at the same time you're installing the code?

Unfortunately, people don't seem to like that answer. They demand that their temporary files should be in /tmp or /var/tmp. For those people, there is no clean answer, so they must choose a hack they can live with.

Make a temporary directory

If you can't use $HOME, the next best answer is to create a private directory to hold your temp file(s), instead of creating the files directly inside a world-writable sandbox like /tmp or /var/tmp. The mktemp command is atomic, and only reports success if it actually created the directory. So long as we do not use the -p option, we can be assured that it actually created a brand new directory, rather than following a symlink to danger.

Here is one example of this approach:

# Bash

i=0 tempdir=
trap '[[ $tempdir ]] && rm -rf "$tempdir"' EXIT

while ((++i <= 10)); do
  tempdir=${TMPDIR:-/tmp}/$RANDOM-$$
  mkdir -m 700 "$tempdir" 2>/dev/null && break
done

if ((i > 10)); then
  echo "Could not create temporary directory" >&2
  exit 1
fi

Instead of RANDOM, awk can be used to generate a random number in a POSIX compatible way:

# POSIX

i=0 tempdir=
trap '[ "$tempdir" ] && rm -rf "$tempdir"' EXIT

while [ $i -lt 10 ]; do
  tempdir=${TMPDIR:-/tmp}/$(awk 'BEGIN { srand (); print rand() }')-$$
  mkdir -m 700 "$temp_dir" 2>/dev/null && break
  sleep 1
  i=$((i+1))
done

if [ $i -ge 10 ]; then
  echo "Could not create temporary directory" >&2
  exit 1
fi

Note however that srand() seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack. (Historical awk implementations that predate POSIX may not even use the time of day for srand(), so don't count on this if you're on an ancient system.)

Some systems have a 14-character filename limit, so avoid the temptation to string $RANDOM together more than twice. You're relying on the atomicity of mkdir for your security, not the obscurity of your random name. If someone fills up /tmp with hundreds of thousands of random-number files to thwart you, you've got bigger issues.

In some systems (like Linux):

  • You have available the mktemp command and you can use its -d option so that it creates temporary directores only accessible for you, with random characters inside their names to make it almost impossible for an attacker to guess the directory name beforehand.

  • You can create filenames longer than 14 characters in /tmp.

# Linux

unset tempdir
trap '[ "$tempdir" ] && rm -rf "$tempdir"' EXIT
tempdir=$(mktemp -d "${TMPDIR:-/tmp}/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX") ||
  { echo "ERROR creating a temporary file" >&2; exit 1; }

And then you can create your particular files inside the temporary directory.

Use platform-specific tools

If you're writing a script for only a specific set of systems, and if those systems have a mktemp or tempfile tool, you can use that. The options will vary from system to system, so you will have to write your script with that in mind. Since some platforms have none of these tools, this is not a portable solution.

Other approaches

Another not-quite-serious suggestion is to include C code in the script that implements a mktemp(1) command based on the mktemp(3) library function, compile it, and use that in the script. But this has a couple problems:

  • The useless Solaris systems where we would need this probably don't have a C compiler either.
  • Chicken and egg problem: we need a temporary file name to hold the compiler's output.

BashFAQ/062 (last edited 2023-04-28 01:07:11 by larryv)