Diff for "BashFAQ/088"

Anchor(faq88)

How can I avoid losing any history lines?

This method is designed to allow you to store a complete log of all commands executed by a friendly user; it is not meant for secure auditing of commands - see [:BashFAQ#faq77:securing bash against history removal].

By default, Bash updates its history only on exit, and it overwrites the existing history with the new version. This prevents you from keeping a complete history log, for two reasons:

• If a user is logged in multiple times, the overwrite will ensure that only the last shell to exit will save its history.
• If your shell terminates abnormally - for example because of network problems, firewall changes or because it was killed - no history will be written.

To solve the first problem, we set the shell option histappend which causes all new history lines to be appended, and ensures that multiple logins do not overwrite each other's history.

To prevent history lines being lost if Bash terminates abnormally, we need to ensure that they are written after each command. We can use the shell builtin history -a to cause an immediate write of all new history lines, and we can automate this execution by adding it to the PROMPT_COMMAND environment variable. This variable contains a command to be executed before any new prompt is shown, and is therefore run after every interactive command is executed.

Note that there are two side effects of running 'history -a' after every command:

• A new login will be able to immediately scroll back through the history of existing logins. So if you wish to run the same command in two sessions, run the command and then initiate the second login and you will be able to retrieve the command immediately.
• More negatively, the history commands of simultaneous interactive shells (for a given user) will be intertwined. Therefore the history is not a guaranteed sequential list of commands as they were executed in a single shell. You may find this confusing if you review the history file as a whole, looking for sections encapsulating particular tasks rather than searching for individual commands. It's probably only an issue if you have multiple people using a single account simultaneously, which is not ideal in any case.

To set all this, use the following in your own ~/.bashrc, or systemwide /etc/bash_profile or /etc/profile to affect all users:

•  HISTFILESIZE=500000
   HISTSIZE=500000
   PROMPT_COMMAND="history -a"
   export HISTFILESIZE HISTSIZE PROMPT_COMMAND
  
   shopt -s histappend

In the above we have also increased the maximum number of lines of history that will be stored in memory, and in the history file. The default is 500 lines, which will cause you to start to lose lines fairly quickly if you are an active user. The value of 500000 is arbitrarily large enough to never be exceeded in most circumstances; you may wish to use a lower figure if you are not interested in older commands. Alternatively, you could use an external log rotator tool to rotate your .bash_history on a time or length basis.

PROMPT_COMMAND may already be used in your system, for example containing control codes to update an XTerm's display bar with your current prompt. If yours is already in use, you can append to it with: PROMPT_COMMAND="$PROMPT_COMMAND ; history -a"

You may also want to set the variables HISTIGNORE and HISTCONTROL to control what is saved, for example to remove duplicate lines - though doing so prevents you from seeing how many times a given command was run by a user, and precisely when (if HISTTIMEFORMAT is also set.)

Finally, note that because PROMPT_COMMAND executes just before a new prompt is printed, you may still lose the last command line if your shell terminates during the execution of this line. As an example, consider: this_cmd_is_never_written_to_history ; kill -9 $$

Once you have enabled these methods, you should find that your history resource becomes much more valuable, allowing you to recall any command you have executed at any time. As such, you should ensure your history file(s) are included in your regular backups.