Size: 5905
Comment: igli: typo extra ' in command (though it still worked in busybox;)
|
← Revision 70 as of 2025-02-09 13:06:43 ⇥
Size: 12150
Comment: remove -- from trap -- const
|
Deletions are marked like this. | Additions are marked like this. |
Line 3: | Line 3: |
Good question. To be filled in later. (Interim hints: {{{tempfile}}} is not portable. {{{mktemp}}} exists more widely, but it may require a {{{-c}}} switch to create the file in advance; or it may create the file by default and barf if {{{-c}}} is supplied. Some systems don't have either command (Solaris, POSIX). There does not appear to be any single command that simply ''works'' everywhere.) The traditional answer has usually been something like this: |
This question is distressingly difficult to answer, because there isn't any single command that simply ''works'' everywhere. This page will discuss the difficulties involved, and will offer a variety of solutions. Which one you choose will depend on your script's needs and expected runtime environment. === The issues === Traditionally, Unix and Unix-like systems have offered one or two world-writable directories (`/tmp` and sometimes `/var/tmp`), where ''all'' programs are expected to create their temporary files. This is convenient for performing cleanups of leftover files, but from a security perspective, it's a ''disaster''. A malicious program can create a file or symbolic link that another program may use unwittingly. Some programs may run as a user account that doesn't have its own private home directory, or any other location where files may be created without disturbance by other programs. These programs have nowhere else to create their files except for `/tmp` or `/var/tmp`. In order to create a temporary file safely in a world-writable directory, a program must perform an ''atomic'' file creation, so that there will not be a [[RaceCondition]] window during which a malicious program may insert its own version of the file. The tools needed to do this in a shell script are not consistently available on all platforms. Some systems have `tempfile`. Others have `mktemp`. Some of the `mktemp` systems (e.g. HP-UX) have a ''broken'' version which doesn't create the file by default -- it simply spits out a name (which is not secure). Such systems may have `mktemp -c` to create the file properly, but the `-c` option isn't recognized on the systems where `mktemp` works properly by default. A few systems (e.g. Solaris) don't have ''either'' `tempfile` or `mktemp`. The creation of a temporary file ''name'' will typically involve the use of a pseudorandom number generator. Some shells offer a `$RANDOM` variable which gives RNG capability, but the POSIX shell specification does not require this to exist. The concatenation of random numbers to create a file name may result in a name that is too long for the file system mounted on `/tmp`. Some extremely old file systems have a 14-character file name length limit. <<Anchor(HOME)>> === Solution 1: Use your home directory. === The best ''portable'' answer is to put your temporary files ''in your home directory'' (or some other private directory, e.g. `$XDG_RUNTIME_DIR`) where nobody else has write access. Then at least you don't have to worry about malicious users. Simplistic PID-based schemes (or hostname + PID for shared file systems) should be enough to prevent conflicts with your own scripts. If you're implementing a daemon which runs under a user account with no home directory, why not simply ''make a private directory for your daemon'' at the same time you're installing the code? Unfortunately, some people don't seem to like that answer. They either don't have the privileges to create a directory for their script to use, or for various reasons they insist that their temporary files must reside in `/tmp` or `/var/tmp`. For those people, there is no clean answer, so they must choose a hack they can live with. <<Anchor(tempdir)>> === Solution 2: Create a private directory at runtime. === If you can't use $HOME, the next best answer is to create a private '''directory''' to hold your temp file(s), instead of creating the files directly inside a world-writable sandbox like `/tmp` or `/var/tmp`. On some systems, the `mktemp` command has a `-d` option to create a directory, rather than a file. If you're fortunate enough to have this option, using it is simple: {{{#!highlight bash # POSIX sh with mktemp -d available tempdir="" cleanup() { test "$tempdir" && rm -rf -- "$tempdir"; } trap cleanup EXIT HUP INT TERM # in Bash you can just trap cleanup EXIT tempdir=$(mktemp -d -- "${TMPDIR:-/tmp}/XXXXXXXXXXXXXX") || { echo 'Error creating a temporary directory' >&2; exit 1; } # Don't forget to call "cleanup" at the end of your script, if you aren't in bash. }}} Then you can create your files inside the temporary directory. If you don't have `mktemp -d`, then the solution becomes more complicated. The `mkdir` command is atomic, and only reports success if it actually creates the directory. So long as we ''do not'' use the `-p` option, we can be assured that it actually creates a brand new directory, rather than following a symlink to danger. Here is one example of this approach: {{{#!highlight bash # Bash tempdir="" trap '[[ $tempdir ]] && rm -rf -- "$tempdir"' EXIT i=0 while ((++i <= 10)); do tempdir=${TMPDIR:-/tmp}/$RANDOM-$$ mkdir -m 700 -- "$tempdir" 2>/dev/null && break done if ((i > 10)); then echo 'Could not create temporary directory' >&2 exit 1 fi }}} Instead of `RANDOM`, `awk` can be used to generate a random number in a POSIX compatible way: {{{#!highlight bash # POSIX sh i=0 tempdir="" cleanup() { [ "$tempdir" ] && rm -rf -- "$tempdir" if [ "$1" != EXIT ]; then trap - "$1" # reset trap, and kill -s "$1" -- "$$" # resend signal to self fi } for sig in EXIT HUP INT TERM; do trap "cleanup $sig" "$sig" done # Generate 10 random 6-digit numbers, separated by commas rand=$( awk 'BEGIN { srand(); for (i = 0; i < 10; i++) printf("%06d,", 1e6*rand()) }' ) while [ "$rand" ]; do tempdir=${TMPDIR:-/tmp}/${rand%%,*}-$$ mkdir -m 700 -- "$tempdir" 2>/dev/null && break rand=${rand#*,} done # if all random numbers have been exhausted if [ -z "$rand" ]; then printf 'Could not create temporary directory\n' >&2 exit 1 fi }}} Note that `srand()` seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack. (Historical awk implementations that predate POSIX may not even use the time of day for `srand()`, so don't count on this if you're on an ancient system.) Some systems have a 14-character filename limit, so avoid the temptation to string `$RANDOM` together more than twice. You're relying on the atomicity of `mkdir` for your security, ''not'' the obscurity of your random name. If someone fills up `/tmp` with hundreds of thousands of random-number files to thwart you, you've got bigger issues. If you don't care for awk's `srand()`, you can try to bring in randomness from other sources. Some systems have a `/dev/random` or `/dev/urandom` device which provides random bytes. These bytes can be turned into a filename: {{{#!highlight bash # POSIX sh with /dev/random available randname() { dd bs=6 count=1 if=/dev/random 2>/dev/null | od -tx1 -An | tr -d ' ' } # Replace the tempdir assignment in the previous example. while [ "$i" -lt 10 ]; do tempdir=${TMPDIR:-/tmp}/$(randname)-$$ }}} If you don't have `/dev/random` or any equivalent interface, you can pull entropy from commands which are likely to give highly volatile output, and use a hash function to turn the entropy into a filename. POSIX doesn't provide a large set of hashing tools, but it does have `cksum`. {{{#!highlight bash # POSIX sh randname() { { date; who; ps -ef; ps auxw; } 2>&1 | cksum | tr -d ' ' | cut -c1-14 } while [ "$i" -lt 10 ]; do tempdir=${TMPDIR:-/tmp}/$(randname)-$$ }}} The use of both `ps` commands tries to ensure that ''at least'' one of them will work. If they both work, that's even better. Feel free to add other highly volatile commands, if your system has them. <<Anchor(nonportable-tools)>> === Solution 3: Use platform-specific tools === If you're writing a script for only a specific set of systems, and if those systems have a `mktemp` or `tempfile` tool which works correctly, you can use that. Make sure that your tool actually ''creates'' the temporary file, and does not simply give you an unused name. The options will vary from system to system, so you will have to write your script with that in mind (see e.g. [[https://unix.stackexchange.com/questions/555058/mktemp-on-macos-not-honouring-tmpdir|MacOS Darwin 2019 notes]]). Since some legacy systems may not have these tools, this is not guaranteed to be a portable solution. But it's often good enough for most operating systems. Here's an example using GNU's `mktemp`: {{{#!highlight bash # POSIX sh tmpfile="" trap '[ "$tmpfile" ] && rm -f -- "$tmpfile"' EXIT HUP INT TERM tmpfile=$(mktemp) }}} The `mktemp` command is the most likely platform-specific tool to exist. It is widely available (GNU coreutils since 2007), and there has been discussion about its status in POSIX (see [[https://unix.stackexchange.com/q/614808|StackExchange 2020]]). However, some old versions of `mktemp` may require a `-c` option to create the file in advance, or they may create the file by default and fail if `-c` is supplied. The `tempfile` is also not portable, but you may find it on some systems that lack `mktemp`. Some legacy systems, like Solaris, may not have either command. In POSIX systems, you could possibly [[#m4|revert to m4]], which has the ability to create a temporary file. However, the command may not be installed by default, and old `m4` versions may be missing this feature. <<Anchor(m4)>> === Solution 4: Use m4. === The `m4` approach, although in POSIX, may not be an ideal solution in practice. For example, the `m4` in MacOS (July 2021) was too old to have the `mkstemp` macro included. Note that `mkstemp` requires a template, including a path prefix, or else it creates the temporary file in the current directory. {{{#!highlight bash # Bash die() { echo "$*" >&2; exit 1; } unset -v tmpfile trap '[ "$tmpfile" ] && rm -f -- "$tmpfile"' EXIT tmpfile=$(m4 - <<< 'mkstemp(/tmp/foo-XXXXXX)') || die "couldn't create temporary file" }}} <<Anchor(manual)>> === Solution 5: Try to create a temporary file manually. === The traditional approach: |
Line 8: | Line 172: |
# Do not use! Race condition! tempfile=/tmp/myname.$$ trap 'rm -f "$tempfile"; exit 1' 1 2 3 15 rm -f "$tempfile" touch "$tempfile" }}} |
# THIS IS A BAD EXAMPLE! DO NOT USE THIS! # Problems: race condition, security tempfile=/tmp/myname.$$ trap 'rm -f -- "$tempfile"; exit 1' HUP INT QUIT TERM rm -f -- "$tempfile" touch -- "$tempfile" }}} |
Line 15: | Line 181: |
---- In some systems (like Linux): * you have available the {{{mktemp}}} command and you can use its {{{-d}}} option so that it creates temporary directores only accessible for you, with random characters inside their names to make it almost impossible for an attacker to guess the directories names beforehand. * you can create filenames longer than 14 characters in'' ''{{{/tmp.}}} * you have available Bash so you can use special features of it. {{{ # Sets $TMPDIR to "/tmp" only if it didn't have a value previously TMPDIR=${TMPDIR:-/tmp} # We can find about $TMPDIR in http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html # Creates a particular temporary directory inside $TMPDIR temporary_dir=$(mktemp -d "$TMPDIR/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX") || { echo "ERROR creating a temporary file" >&2; exit 1; } # Remove the temporary directory when the script finishes, or when it receives a signal trap 'rm -rf "$temporary_dir"' 0 # remove directory when script finishes trap 'exit 2' 1 2 3 15 # terminate script when receiving signal }}} And then you can create your particular files inside the temporary folder. ''If you want to make life more difficult to an adversary and you are using Bash, you can use random numbers in the names of your temporary files to prevent strange cases like when you are using a shared system, your program is paused for a long time, its process ID is known, it uses a temporary file, root runs Tmpwatch (or similar) to delete temporary files that are not used for a long time and an adversary creates a replica of your temporary folder (if you use random names the adversary will be able to know the name of your temporary folder but not the name of your files inside it)''. For example: {{{ # Prepares the name of the future temporary file temporary_file="$temporary_dir/strings-$RANDOM-$RANDOM-$RANDOM" # Then you can use the temporary file, like in grep string file > "$temporary_file" }}} ---- A different suggestion (remove if not universal): A temporary directory can be created that is unlikely to match an existing directory using the {{{RANDOM}}} variable as follows: {{{ temp_dir=/tmp/$RANDOM mkdir "$temp_dir" }}} This will make a directory of the form: {{{/tmp/20445/}}}. To decrease the chance of collision with an existing directory, the {{{RANDOM}}} variable can be used a number of times: {{{ temp_dir=/tmp/$RANDOM-$RANDOM-$RANDOM mkdir "$temp_dir" }}} This will make a directory of the form: {{{/tmp/24953-2875-2182/}}} . This avoids a race condition because the `mkdir` is atomic, as we see in [[BashFAQ/045|FAQ #45]]. . ''Hmmm... this has potential, if you check the exit status of `mkdir` to be sure it actually created the directory. And set `umask` to something fairly restrictive as well. It could use some more peer review, though. -- GreyCat'' . ''Oh, also, you shouldn't assume you can create filenames longer than 14 characters in `/tmp`. There are still some systems out there with 14-character filename limits. --GreyCat'' . ''Also also, {{{RANDOM}}} is not available in the Bourne shell, so it still fails on Solaris. This is why most legacy Bourne shell scripts use {{{$$}}} for that purpose.'' --GreyCat |
To make this work safely, we must perform an ''atomic file creation''. The way to do this in a shell script is to turn on the ''noclobber'' option (`set -C`) and then attempt an output redirection. If the redirection fails, try again with a different name, until it succeeds, or until too many tries have already been attempted. {{{#!highlight bash # POSIX sh mktmpfile() { ( umask 077; set -C i=0 while test $i -lt 10; do f=${TMPDIR:-/tmp}/ME-$$-$i ( : > "$f" ) >/dev/null 2>&1 && { printf '%s\n' "$f"; exit 0; } i=$((i+1)) done exit 1 ) } tmpfile= trap 'test "$tmpfile" && rm -f -- "$tmpfile"' EXIT HUP INT TERM tmpfile=$(mktmpfile) || { echo >&2 'Error creating temp file'; exit 1; } }}} If `$RANDOM` or some other source of random numbers is available, you may use that when generating the file names. The `randname()` functions from [[#tempdir|Solution 2]] may also be used here, for example. <<Anchor(other)>> === Solution 6: Use other approaches. === |
Line 65: | Line 213: |
---- Instead of {{{RANDOM}}}, `awk` can be used to generate a random number in a POSIX compatible way: {{{ temp_dir=/tmp/$(awk 'BEGIN { srand (); print rand() }') mkdir -m 700 "$temp_dir" }}} Note, however that "srand()" seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack. ---- Combining the above we can use: {{{ temp_dir=/tmp/"$(awk -v p=$$ 'BEGIN { srand(); s = rand(); sub(/^0./, "", s); printf("%X_%X", p, s) }')" mkdir -m 700 "$temp_dir" || { echo '!! unable to create a tempdir' >&2; exit 1; } }}} Since the default awk numeric format is %g which equates to 6 digits after the decimal point, we have a 6 digit trail. 999,999 is F423F which gives us one character back, another for the {{{_}}} and 8 available for a prefixed pid on a 14-char filesystem. An unsigned 32-bit pid will be a maximum of 8 hex digits, so the above should stand a chance in most places one might encounter such an fs. And of course we bail out as greycat mentioned when the creation fails. |
How do I create a temporary file in a secure manner?
This question is distressingly difficult to answer, because there isn't any single command that simply works everywhere. This page will discuss the difficulties involved, and will offer a variety of solutions. Which one you choose will depend on your script's needs and expected runtime environment.
The issues
Traditionally, Unix and Unix-like systems have offered one or two world-writable directories (/tmp and sometimes /var/tmp), where all programs are expected to create their temporary files. This is convenient for performing cleanups of leftover files, but from a security perspective, it's a disaster. A malicious program can create a file or symbolic link that another program may use unwittingly.
Some programs may run as a user account that doesn't have its own private home directory, or any other location where files may be created without disturbance by other programs. These programs have nowhere else to create their files except for /tmp or /var/tmp.
In order to create a temporary file safely in a world-writable directory, a program must perform an atomic file creation, so that there will not be a RaceCondition window during which a malicious program may insert its own version of the file. The tools needed to do this in a shell script are not consistently available on all platforms. Some systems have tempfile. Others have mktemp. Some of the mktemp systems (e.g. HP-UX) have a broken version which doesn't create the file by default -- it simply spits out a name (which is not secure). Such systems may have mktemp -c to create the file properly, but the -c option isn't recognized on the systems where mktemp works properly by default. A few systems (e.g. Solaris) don't have either tempfile or mktemp.
The creation of a temporary file name will typically involve the use of a pseudorandom number generator. Some shells offer a $RANDOM variable which gives RNG capability, but the POSIX shell specification does not require this to exist.
The concatenation of random numbers to create a file name may result in a name that is too long for the file system mounted on /tmp. Some extremely old file systems have a 14-character file name length limit.
Solution 1: Use your home directory.
The best portable answer is to put your temporary files in your home directory (or some other private directory, e.g. $XDG_RUNTIME_DIR) where nobody else has write access. Then at least you don't have to worry about malicious users. Simplistic PID-based schemes (or hostname + PID for shared file systems) should be enough to prevent conflicts with your own scripts.
If you're implementing a daemon which runs under a user account with no home directory, why not simply make a private directory for your daemon at the same time you're installing the code?
Unfortunately, some people don't seem to like that answer. They either don't have the privileges to create a directory for their script to use, or for various reasons they insist that their temporary files must reside in /tmp or /var/tmp. For those people, there is no clean answer, so they must choose a hack they can live with.
Solution 2: Create a private directory at runtime.
If you can't use $HOME, the next best answer is to create a private directory to hold your temp file(s), instead of creating the files directly inside a world-writable sandbox like /tmp or /var/tmp. On some systems, the mktemp command has a -d option to create a directory, rather than a file. If you're fortunate enough to have this option, using it is simple:
1 # POSIX sh with mktemp -d available
2
3 tempdir=""
4 cleanup() { test "$tempdir" && rm -rf -- "$tempdir"; }
5 trap cleanup EXIT HUP INT TERM # in Bash you can just trap cleanup EXIT
6 tempdir=$(mktemp -d -- "${TMPDIR:-/tmp}/XXXXXXXXXXXXXX") ||
7 { echo 'Error creating a temporary directory' >&2; exit 1; }
8
9 # Don't forget to call "cleanup" at the end of your script, if you aren't in bash.
Then you can create your files inside the temporary directory.
If you don't have mktemp -d, then the solution becomes more complicated. The mkdir command is atomic, and only reports success if it actually creates the directory. So long as we do not use the -p option, we can be assured that it actually creates a brand new directory, rather than following a symlink to danger.
Here is one example of this approach:
1 # Bash
2
3 tempdir=""
4 trap '[[ $tempdir ]] && rm -rf -- "$tempdir"' EXIT
5
6 i=0
7 while ((++i <= 10)); do
8 tempdir=${TMPDIR:-/tmp}/$RANDOM-$$
9 mkdir -m 700 -- "$tempdir" 2>/dev/null && break
10 done
11
12 if ((i > 10)); then
13 echo 'Could not create temporary directory' >&2
14 exit 1
15 fi
Instead of RANDOM, awk can be used to generate a random number in a POSIX compatible way:
1 # POSIX sh
2
3 i=0
4 tempdir=""
5 cleanup() {
6 [ "$tempdir" ] && rm -rf -- "$tempdir"
7 if [ "$1" != EXIT ]; then
8 trap - "$1" # reset trap, and
9 kill -s "$1" -- "$$" # resend signal to self
10 fi
11 }
12 for sig in EXIT HUP INT TERM; do
13 trap "cleanup $sig" "$sig"
14 done
15
16 # Generate 10 random 6-digit numbers, separated by commas
17 rand=$(
18 awk 'BEGIN { srand(); for (i = 0; i < 10; i++) printf("%06d,", 1e6*rand()) }'
19 )
20 while [ "$rand" ]; do
21 tempdir=${TMPDIR:-/tmp}/${rand%%,*}-$$
22 mkdir -m 700 -- "$tempdir" 2>/dev/null && break
23 rand=${rand#*,}
24 done
25
26 # if all random numbers have been exhausted
27 if [ -z "$rand" ]; then
28 printf 'Could not create temporary directory\n' >&2
29 exit 1
30 fi
Note that srand() seeds the random number generator using seconds since the epoch which is fairly easy for an adversary to predict and perform a denial of service attack. (Historical awk implementations that predate POSIX may not even use the time of day for srand(), so don't count on this if you're on an ancient system.)
Some systems have a 14-character filename limit, so avoid the temptation to string $RANDOM together more than twice. You're relying on the atomicity of mkdir for your security, not the obscurity of your random name. If someone fills up /tmp with hundreds of thousands of random-number files to thwart you, you've got bigger issues.
If you don't care for awk's srand(), you can try to bring in randomness from other sources. Some systems have a /dev/random or /dev/urandom device which provides random bytes. These bytes can be turned into a filename:
If you don't have /dev/random or any equivalent interface, you can pull entropy from commands which are likely to give highly volatile output, and use a hash function to turn the entropy into a filename. POSIX doesn't provide a large set of hashing tools, but it does have cksum.
The use of both ps commands tries to ensure that at least one of them will work. If they both work, that's even better. Feel free to add other highly volatile commands, if your system has them.
Solution 3: Use platform-specific tools
If you're writing a script for only a specific set of systems, and if those systems have a mktemp or tempfile tool which works correctly, you can use that. Make sure that your tool actually creates the temporary file, and does not simply give you an unused name. The options will vary from system to system, so you will have to write your script with that in mind (see e.g. MacOS Darwin 2019 notes). Since some legacy systems may not have these tools, this is not guaranteed to be a portable solution. But it's often good enough for most operating systems.
Here's an example using GNU's mktemp:
The mktemp command is the most likely platform-specific tool to exist. It is widely available (GNU coreutils since 2007), and there has been discussion about its status in POSIX (see StackExchange 2020). However, some old versions of mktemp may require a -c option to create the file in advance, or they may create the file by default and fail if -c is supplied.
The tempfile is also not portable, but you may find it on some systems that lack mktemp.
Some legacy systems, like Solaris, may not have either command.
In POSIX systems, you could possibly revert to m4, which has the ability to create a temporary file. However, the command may not be installed by default, and old m4 versions may be missing this feature.
Solution 4: Use m4.
The m4 approach, although in POSIX, may not be an ideal solution in practice. For example, the m4 in MacOS (July 2021) was too old to have the mkstemp macro included. Note that mkstemp requires a template, including a path prefix, or else it creates the temporary file in the current directory.
Solution 5: Try to create a temporary file manually.
The traditional approach:
# THIS IS A BAD EXAMPLE! DO NOT USE THIS! # Problems: race condition, security tempfile=/tmp/myname.$$ trap 'rm -f -- "$tempfile"; exit 1' HUP INT QUIT TERM rm -f -- "$tempfile" touch -- "$tempfile"
The problem with this is: if the file already exists (for example, as a symlink to /etc/passwd), then the script may write things in places they should not be written. Even if you remove the file immediately before using it, you still have a RaceCondition: someone could re-create a malicious symlink in the interval between your shell commands.
To make this work safely, we must perform an atomic file creation. The way to do this in a shell script is to turn on the noclobber option (set -C) and then attempt an output redirection. If the redirection fails, try again with a different name, until it succeeds, or until too many tries have already been attempted.
1 # POSIX sh
2
3 mktmpfile() {
4 (
5 umask 077; set -C
6 i=0
7 while test $i -lt 10; do
8 f=${TMPDIR:-/tmp}/ME-$$-$i
9 ( : > "$f" ) >/dev/null 2>&1 && { printf '%s\n' "$f"; exit 0; }
10 i=$((i+1))
11 done
12 exit 1
13 )
14 }
15
16 tmpfile=
17 trap 'test "$tmpfile" && rm -f -- "$tmpfile"' EXIT HUP INT TERM
18 tmpfile=$(mktmpfile) || { echo >&2 'Error creating temp file'; exit 1; }
If $RANDOM or some other source of random numbers is available, you may use that when generating the file names. The randname() functions from Solution 2 may also be used here, for example.
Solution 6: Use other approaches.
Another not-quite-serious suggestion is to include C code in the script that implements a mktemp(1) command based on the mktemp(3) library function, compile it, and use that in the script. But this has a couple problems:
- The useless Solaris systems where we would need this probably don't have a C compiler either.
- Chicken and egg problem: we need a temporary file name to hold the compiler's output.