BASH Frequently Asked Questions
Note: The FAQ was split into individual pages for easier editing. Also, for faster loading of this page, the answers are no longer presented here in their entirety. |
These are answers to frequently asked questions on channel #bash on the freenode IRC network. These answers are contributed by the regular members of the channel (originally heiner, and then others including greycat and r00t), and by users like you. If you find something inaccurate or simply misspelled, please feel free to correct it!
All the information here is presented without any warranty or guarantee of accuracy. Use it at your own risk. When in doubt, please consult the man pages or the GNU info pages as the authoritative references.
BASH is a BourneShell compatible shell, which adds many new features to its ancestor. Most of them are available in the KornShell, too. The answers given in this FAQ may be slanted toward Bash, or they may be slanted toward the lowest common denominator Bourne shell, depending on who wrote the answer. In most cases, an effort is made to provide both a portable (Bourne) and an efficient (Bash, where appropriate) answer. If a question is not strictly shell specific, but rather related to Unix, it may be in the UnixFaq.
This FAQ assumes a certain level of familiarity with basic shell script syntax. If you're completely new to Bash or to the Bourne family of shells, you may wish to start with the (incomplete) BashGuide.
If you can't find the answer you're looking for here, try BashPitfalls. If you want to help, you can add new questions with answers here, or try to answer one of the BashOpenQuestions.
Chet Ramey's official Bash FAQ contains many technical questions not covered here.
Contents
- How can I read a file (data stream, variable) line-by-line?
- How can I store the return value/output of a command in a variable?
- How can I insert a blank character after each character?
- How can I check whether a directory is empty or not? How do I check for any *.mpg files?
- How can I use array variables?
- How can I use variable variables (indirect variables, pointers, references) or associative arrays?
- Is there a function to return the length of a string?
- How can I recursively search all files for a string?
- My command line produces no output: tail -f logfile | grep 'foo bar'
- How can I recreate a directory hierarchy structure, without the files?
- How can I print the n'th line of a file?
- How do I invoke a shell command from a non-shell application?
- How can I concatenate two variables? How do I append a string to a variable?
- How can I redirect the output of multiple commands at once?
- How can I run a command on all files with the extension .gz?
- How can I use a logical AND/OR/NOT in a shell pattern (glob)?
- How can I group expressions, e.g. (A AND B) OR C?
- How can I use numbers with leading zeros in a loop, e.g. 01, 02?
- How can I split a file into line ranges, e.g. lines 1-10, 11-20, 21-30?
- How can I replace a string with another string in all files?
- How can I calculate with floating point numbers instead of just integers?
- I want to launch an interactive shell that has a special set of aliases and functions, not the ones in the user's ~/.bashrc.
- I set variables in a loop. Why do they suddenly disappear after the loop terminates? Or, why can't I pipe data to read?
- How can I access positional parameters after $9?
- How can I randomize (shuffle) the order of lines in a file? (Or select a random line from a file, or select a random file from a directory.)
- How can two unrelated processes communicate?
- How do I determine the location of my script? I want to read some config files from the same place.
- How can I display the target of a symbolic link?
- How can I rename all my *.foo files to *.bar, or convert spaces to underscores, or convert upper-case file names to lower case?
- What is the difference between test, [ and [[ ?
- How can I redirect the output of 'time' to a variable or file?
- How can I find a process ID for a process given its name?
- Can I do a spinner in Bash?
- How can I handle command-line arguments to my script easily?
- How can I get all lines that are: in both of two files (set intersection) or in only one of two files (set subtraction).
- How can I print text in various colors?
- How do Unix file permissions work?
- What are all the dot-files that bash reads?
- How do I use dialog to get input from the user?
- How do I determine whether a variable contains a substring?
- How can I find out if a process is still running?
- Why does my crontab job fail? 0 0 * * * some command > /var/log/mylog.`date +%Y%m%d`
- How do I create a progress bar?
- How can I ensure that only one instance of a script is running at a time (mutual exclusion)?
- I want to check to see whether a word is in a list (or an element is a member of a set).
- How can I redirect stderr to a pipe?
- Eval command and security issues
- How can I view periodic updates/appends to a file? (ex: growing log file)
- I'm trying to put a command in a variable, but the complex cases always fail!
- I want history-search just like in tcsh. How can I bind it to the up and down keys?
- How do I convert a file from DOS format to UNIX format (remove CRs from CR-LF line terminators)?
- I have a fancy prompt with colors, and now bash doesn't seem to know how wide my terminal is. Lines wrap around incorrectly.
- How can I tell whether a variable contains a valid number?
- Tell me all about 2>&1 -- what's the difference between 2>&1 >foo and >foo 2>&1, and when do I use which?
- How can I untar or unzip multiple tarballs at once?
- How can group entries (in a file by common prefixes)?
- Can bash handle binary data?
- I saw this command somewhere: :(){ :|:& } (fork bomb). How does it work?
- I'm trying to write a script that will change directory (or set a variable), but after the script finishes, I'm back where I started (or my variable isn't set)!
- Is there a list of which features were added to specific releases (versions) of Bash?
- How do I create a temporary file in a secure manner?
- My ssh client hangs when I try to run a remote background job!
- Why is it so hard to get an answer to the question that I asked in #bash?
- Is there a "PAUSE" command in bash like there is in MSDOS batch scripts? To prompt the user to press any key to continue?
- I want to check if [[ $var == foo || $var == bar || $var == more ]] without repeating $var n times.
- How can I trim leading/trailing white space from one of my variables?
- How do I run a command, and have it abort (timeout) after N seconds?
- I want to automate an ssh (or scp, or sftp) connection, but I don't know how to send the password....
- How do I convert Unix (epoch) timestamps to human-readable values?
- How do I convert an ASCII character to its decimal (or hexadecimal) value and back?
- How can I ensure my environment is configured for cron, batch, and at jobs?
- How can I use parameter expansion? How can I get substrings? How can I get a file without its extension, or get just a file's extension?
- How do I get the effects of those nifty Bash Parameter Expansions in older shells?
- How do I use 'find'? I can't understand the man page at all!
- How do I get the sum of all the numbers in a column?
- How do I log history or "secure" bash against history removal?
- I want to set a user's password using the Unix passwd command, but how do I script that? It doesn't read standard input!
- How can I grep for lines containing foo AND bar, foo OR bar? Or for files containing foo AND bar, possibly on separate lines?
- How can I make an alias that takes an argument?
- How can I determine whether a command exists anywhere in my PATH?
- Why is $(...) preferred over `...` (backticks)?
- How do I determine whether a variable is already defined? Or a function?
- How do I return a string (or large number, or negative number) from a function? "return" only lets me give a number from 0 to 255.
- How to write several times to a fifo without having to reopen it?
- How to ignore aliases or functions when running a command?
- How can I get the permissions of a file without parsing ls -l output?
- How can I avoid losing any history lines?
- I'm using a loop which runs once per line of input but it only seems to run once; everything after the first line is ignored?
- How do I prepend a text to a file (the opposite of >>)?
- I'm trying to get the number of columns or lines of my terminal but the variables COLUMNS / LINES are always empty
- How do I write a CGI script that accepts parameters?
- How can I see a progress bar when copying/moving files?
- I want to get an alert when my disk is full (parsing df output).
- I'm getting "Argument list too long". How can I process a large list in chunks?
- ssh eats my word boundaries! I can't do ssh remotehost make CFLAGS="-g -O"!
- How do I determine whether a symlink is dangling (broken)?
- How to add localization support to your bash scripts
- How can I get the newest (or oldest) file from a directory?
- How do I do string manipulations in bash?
1. How can I read a file (data stream, variable) line-by-line?
Use a while loop and the read command:
2. How can I store the return value/output of a command in a variable?
Well, that depends whether you want to store the command's output (either stdout, or stdout + stderr) or its exit status (0 to 255, with 0 typically meaning "success").
3. How can I insert a blank character after each character?
sed 's/./& /g'
4. How can I check whether a directory is empty or not? How do I check for any *.mpg files?
In Bash, you can do this safely and easily with the nullglob and dotglob options (which change the behaviour of globbing), and arrays:
5. How can I use array variables?
BASH and KornShell have one-dimensional arrays indexed by a numerical expression, e.g.:
6. How can I use variable variables (indirect variables, pointers, references) or associative arrays?
There are two halves to this: evaluating variables, and assigning values. We'll take each half separately:
7. Is there a function to return the length of a string?
The fastest way, not requiring external programs (but usable only with BASH and KornShell):
8. How can I recursively search all files for a string?
90% of the time, all you need is one of these:
9. My command line produces no output: tail -f logfile | grep 'foo bar'
Most standard Unix commands buffer their output if used non-interactively. This means, that they don't write each character (or even each line) as they are ready, but collect a larger number (e.g. 4 kilobytes) before printing it. In the case above, the tail command buffers its output, and therefore grep only gets its input in e.g. 4K blocks.
10. How can I recreate a directory hierarchy structure, without the files?
With the cpio program:
11. How can I print the n'th line of a file?
The dirty (but not quick) way would be:
12. How do I invoke a shell command from a non-shell application?
13. How can I concatenate two variables? How do I append a string to a variable?
There is no (explicit) concatenation operator for strings (either literal or variable dereferences) in the shell; you just write them adjacent to each other:
14. How can I redirect the output of multiple commands at once?
Redirecting the standard output of a single command is as easy as
15. How can I run a command on all files with the extension .gz?
Often a command already accepts several files as arguments, e.g.
16. How can I use a logical AND/OR/NOT in a shell pattern (glob)?
"Globs" are simple patterns that can be used to match filenames or strings. They're generally not very powerful. If you need more power, there are a few options available.
17. How can I group expressions, e.g. (A AND B) OR C?
The TestCommand [ uses parentheses () for expression grouping. Given that "AND" is "-a", and "OR" is "-o", the following expression
18. How can I use numbers with leading zeros in a loop, e.g. 01, 02?
As always, there are different ways to solve the problem, each with its own advantages and disadvantages.
19. How can I split a file into line ranges, e.g. lines 1-10, 11-20, 21-30?
Some Unix systems provide the split utility for this purpose:
20. How can I replace a string with another string in all files?
sed is a good command to replace strings, e.g.
21. How can I calculate with floating point numbers instead of just integers?
BASH does not have built-in floating point arithmetic:
22. I want to launch an interactive shell that has a special set of aliases and functions, not the ones in the user's ~/.bashrc.
Just specify a different start-up file:
23. I set variables in a loop. Why do they suddenly disappear after the loop terminates? Or, why can't I pipe data to read?
24. How can I access positional parameters after $9?
Use ${10} instead of $10. This works for BASH and KornShell, but not for older BourneShell implementations. Another way to access arbitrary positional parameters after $9 is to use for, e.g. to get the last parameter:
25. How can I randomize (shuffle) the order of lines in a file? (Or select a random line from a file, or select a random file from a directory.)
To randomize the lines of a file, here is one approach. This one involves generating a random number, which is prefixed to each line; then sorting the resulting lines, and removing the numbers.
26. How can two unrelated processes communicate?
Two unrelated processes cannot use the arguments, the environment or stdin/stdout to communicate; some form of inter-process communication (IPC) is required.
27. How do I determine the location of my script? I want to read some config files from the same place.
This topic comes up frequently. This answer covers not only the expression used above ("configuration files"), but also several variant situations. If you've been directed here, please read this entire answer before dismissing it.
28. How can I display the target of a symbolic link?
The nonstandard external command readlink(1) can be used to display the target of a symbolic link: