Diff for "BashFAQ/030"

How can I rename all my *.foo files to *.bar, or convert spaces to underscores, or convert upper-case file names to lower case?

There are a bunch of different ways to do this, depending on which nonstandard tools you have available. Even with just standard POSIX tools, you can still perform most of the simple cases. We'll show the portable tool examples first.

You can do most non-recursive mass renames with a loop and some Parameter Expansions, like this:

# POSIX
# Rename all *.foo to *.bar
for f in *.foo; do mv -- "$f" "${f%.foo}.bar"; done

To check what the command would do without actually doing it, you can add an echo before the mv. This applies to almost(?) every example on this page, so we won't mention it again.

# POSIX
# This removes the extension .zip from all the files.
for file in ./*.zip; do mv "$file" "${file%.zip}"; done

The "--" and "./*" are to protect from problematic filenames that begin with "-". You only need one or the other, not both, so pick your favorite.

Here are some similar examples, using Bash-specific parameter expansions:

# Bash
# Replace all spaces with underscores
for f in *\ *; do mv -- "$f" "${f// /_}"; done

For more techniques on dealing with files with inconvenient characters in their names, see FAQ #20.

# Bash
# Replace "foo" with "bar", even if it's not the extension
for file in ./*foo*; do mv "$file" "${file//foo/bar}"; done

All the above examples invoke the external command mv(1) once for each file, so they may not be as efficient as some of the nonstandard implementations.

Recursively

If you want to rename files recursively, then it becomes much more challenging. This example renames *.foo to *.bar:

# Bash
# Also requires GNU or BSD find(1)
# Recursively change all *.foo files to *.bar

find . -type f -name '*.foo' -print0 | while IFS= read -r -d '' f; do
  mv -- "$f" "${f%.foo}.bar"
done

This example uses Bash 4's globstar instead of GNU find:

# Bash 4
# Replace "foo" with "bar" in all files recursively.
# "foo" must NOT appear in a directory name!

shopt -s globstar
for file in /path/to/**/*foo*; do
    mv -- "$file" "${file//foo/bar}"
done

The trickiest part of recursive renames is ensuring that you do not change the directory component of a pathname, because something like this is doomed to failure:

mv "./FOO/BAR/FILE.TXT" "./foo/bar/file.txt"

Therefore, any recursive renaming command should only change the filename component of each pathname, like this:

mv "./FOO/BAR/FILE.TXT" "./FOO/BAR/file.txt"

If you need to rename the directories as well, those should be done separately. Furthermore, recursive directory renaming should either be done depth-first (changing only the last component of the directory name in each instance), or in several passes. Depth-first works better in the general case.

Here's an example script that uses depth-first recursion (changes spaces in names to underscores, but you just need to change the ren() function to do anything you want) to rename both files and directories. Again, it's easy to modify to make it act only on files or only on directories, or to act only on files with a certain extension, to avoid or force overwriting files, etc.:

# Bash
ren() {
  local newname
  newname=${1// /_}
  [[ $1 != "$newname" ]] && mv -- "$1" "$newname"
}

traverse() {
  local file
  cd -- "$1" || exit
  for file in *; do
    [[ -d $file ]] && traverse "$file"
    ren "$file"
  done
  cd .. || exit
}

# main program
shopt -s nullglob
traverse /path/to/startdir

Here is another way to recursively rename all directories and files with spaces in their names:

find . -depth -name "* *" -exec bash -c 'dir=${1%/*} base=${1##*/}; mv "$1" "$dir/${base// /_}"' _ {} \;

or, if your version of find accepts it, this is more efficient as it runs one bash for many files instead of one bash per file:

find . -depth -name "* *" -exec bash -c 'for f; do dir=${f%/*} base=${f##*/}; mv "$f" "$dir/${base// /_}"; done' _ {} +

Upper- and lower-case

To convert filenames to lower-case with only standard tools, you need something that can take a mixed-case filename as input and give back the lowercase version as output. In Bash 4 and higher, there is a parameter expansion that can do it:

# Bash 4
for f in *[[:upper:]]*; do mv -- "$f" "${f,,}"; done

Otherwise, tr(1) may be helpful:

# tolower - convert file names to lower case
# POSIX
for file do
    [ -f "$file" ] || continue                # ignore non-existing names
    newname=$(printf %s "$file" | tr '[:upper:]' '[:lower:]')     # lower case
    [ "$file" = "$newname" ] && continue      # nothing to do
    [ -f "$newname" ] && continue             # don't overwrite existing files
    mv -- "$file" "$newname"
done

This example will not handle filenames that end with newlines, because the CommandSubstitution will eat them. The workaround for that is to append a character in the command substitution, and remove it afterward. Thus:

newname=$(printf %sx "$file" | tr '[:upper:]' '[:lower:]')
newname=${newname%x}

We use the fancy range notation, because tr can behave very strangely when using the A-Z range on some locales:

imadev:~$ echo Hello | tr A-Z a-z
hÉMMÓ

To make sure you aren't caught by surprise when using tr with ranges, either use the fancy range notations, or set your locale to C.

imadev:~$ echo Hello | LC_ALL=C tr A-Z a-z
hello
imadev:~$ echo Hello | tr '[:upper:]' '[:lower:]'
hello
# Either way is fine here.

Note that GNU tr doesn't support multi-byte characters (like non-ASCII UTF-8 ones). So on GNU systems, you may prefer:

# GNU
sed 's/.*/\L&/g'
# POSIX
awk '{print tolower($0)}'

This technique can also be used to replace all unwanted characters in a file name, e.g. with '_' (underscore). The script is the same as above, with only the "newname=..." line changed.

# renamefiles - rename files whose name contain unusual characters
# POSIX
for file do
    [ -f "$file" ] || continue            # ignore non-regular files, etc.
    newname=$(printf '%s\n' "$file" | sed 's/[^[:alnum:]_.]/_/g' | paste -sd _ -)
    [ "$file" = "$newname" ] && continue  # nothing to do
    [ -f "$newname" ] && continue         # do not overwrite existing files
    mv -- "$file" "$newname"
done

The character class in [] contains all the characters we want to keep (after the ^); modify it as needed. The [:alnum:] range stands for all the letters and digits of the current locale. Note however that it will not replace bytes that don't form valid characters (like characters encoded in the wrong character set).

Here's an example that does the same thing, but this time using Parameter Expansion instead of sed:

# renamefiles (more efficient, less portable version)
# Bash/Ksh/Zsh
for file do
   [[ -f $file ]] || continue
   newname=${file//[![:alnum:]_.]/_}
   [[ $file = "$newname" ]] && continue
   [[ -e $newname ]] && continue
   [[ -L $newname ]] && continue
   mv -- "$file" "$newname"
done

It should be noted that all these examples contain a race condition -- an existing file could be overwritten if it is created in between the [ -e "$newname" ... and mv "$file" ... commands. Solving this issue is beyond the scope of this page, however adding the -i and (GNU specific) -T option to mv can reduce its impact.

One final note about changing the case of filenames: when using GNU mv, on many file systems, attempting to rename a file to its lowercase or uppercase equivalent will fail. (This applies to Cygwin on DOS/Windows systems using FAT or NTFS file systems; to GNU mv on Mac OS X systems using HFS+ in case-insensitive mode; as well as to Linux systems which have mounted Windows/Mac file systems, and possibly many other setups.) GNU mv checks both the target names before attempting a rename, and due to the file system's mapping, it thinks that the destination "already exists":

mv README Readme    # fails with GNU mv on FAT file systems, etc.

The workaround for this is to rename the file twice: first to a temporary name which is completely different from the original name, then to the desired name.

mv README tempfilename &&
mv tempfilename Readme

Nonstandard tools

To convert filenames to lower case, if you have the utility mmv(1) on your machine, you could simply do:

# convert all filenames to lowercase
mmv "*" "#l1"

Some GNU/Linux distributions have a rename(1) command; however, the syntax differs from one distribution to the next. Debian uses the perl rename script (formerly included with Perl; now it is not), which it installs as prename(1) and rename(1). Red Hat uses a totally different rename(1) command.

The prename script is extremely flexible. For example, it can be used to change files to lower-case:

# convert all filenames to lowercase
prename '$_=lc($_)' ./*

Alternatively, you can also use:

# convert all filenames to lowercase
prename 'y/A-Z/a-z/' ./*

For prename to use Unicode instead of ASCII for files encoded in UTF-8:

# convert all filenames to lowercase using Unicode rules
PERL_UNICODE=SA rename '$_=lc' ./*

To assume the current locale charset for filenames:

rename 'BEGIN{use Encode::Locale qw(decode_argv);decode_argv} $_=lc'

(note that it still doesn't use the locale's rules for case conversion. For instance, in a Turkish locale, I would be converted to i, not ı).

Or recursively:

# convert all filenames to lowercase, recursively (assumes a find
# implementation with support for the non-standard -execdir predicate)
#
# Note: this will not change directory names. That's because -execdir
# cd's to the parent directory before running the command. That means
# however that (despite the +), one prename command is executed for
# each file to rename.
find . -type f -name '*[[:upper:]]*' -execdir prename '$_=lc($_)' {} +

A more efficient and portable approach:

find . -type f -name '*[[:upper:]]*' -exec prename 's{[^/]*$}{lc($&)}e' {} +

Or to replace all underscores with spaces:

prename 's/_/ /g' ./*_*

CategoryShell