Differences between revisions 3 and 4
Revision 3 as of 2012-05-18 02:34:18
Size: 2125
Editor: long01cpe47-51
Comment: Added one-liner that uses awk.
Revision 4 as of 2012-05-18 02:36:57
Size: 2121
Editor: EricPruitt
Comment: Removed stray word from my earlier edit.
Deletions are marked like this. Additions are marked like this.
Line 37: Line 37:
awk '{system("printf \"`date +%T ` \" >&2")}$0' awk '{system("printf \"`date +%T ` \">&2")}$0'
Line 40: Line 40:
If the timestamps are printed to stderr using the ">&2", so remove those three characters if you want the timestamps in stdout. The timestamps are printed to stderr using the ">&2", so remove those three characters if you want the timestamps in stdout.

How do I add a timestamp to every line of a stream?

There are numerous ways to do this, but all of them are either limited by the available tools, or slow. We'll show a few examples.

Let's start with the slow, portable way first and get it over with:

# POSIX
while IFS= read -r line; do
  echo "$(date +%Y%m%d-%H:%M:%S) $line"
done

The obvious disadvantage to this is that we are forking a subshell, and then executing the external date command, for every line of input. If we only get a line every couple seconds, that may be acceptable. But if we're trying to timestamp a stream that gets dozens of lines per second, we may not even be able to keep up with the writer.

There are various ways to do it without forking for every line, but they all require nonstandard tools or specific shells. Bash 4.2 can do it with printf:

# Bash 4.2
while read -r; do
  printf "%(%Y%m%d-%H:%M:%S)T %s\n" -1 "$REPLY"
done

The %(...)T format specifier is new in bash 4.2. The argument of -1 tells it to use the current time, rather than a time passed as an argument. See the man page for details.

Another way is to write a perl one-liner:

perl -p -e '@l=localtime; printf "%04d%02d%02d-%02d:%02d:%02d ", 1900+$l[5], $l[4], $l[3], $l[2], $l[1], $l[0]'

I'm sure someone will come up with a 7-byte alternative that does the same thing using some magic perl syntax I've never seen before and can't understand....

Here's another variant that uses awk and the date command:

awk '{system("printf \"`date +%T ` \">&2")}$0'

The timestamps are printed to stderr using the ">&2", so remove those three characters if you want the timestamps in stdout.

There are other tools available specifically for timestamping logfiles and the like. One of them is multilog from daemontools; but its timestamping format is TAI64N which is not human-readable. Another is ts from the moreutils package.

BashFAQ/107 (last edited 2020-12-21 19:30:23 by GreyCat)