Differences between revisions 34 and 48 (spanning 14 versions)
Revision 34 as of 2012-02-21 06:52:33
Size: 7983
Editor: 125
Comment:
Revision 48 as of 2015-06-20 22:13:20
Size: 10413
Comment: [ -f ] only tests if the file is a regular file.
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
Some GNU/Linux distributions have a {{{rename(1)}}} command, which you can use for the former; however, the syntax differs from one distribution to the next, so it's not a portable answer....

Consult your system's man pages if you want to learn how to use your {{{rename}}} command, if you have one at all. It's often perfectly good for one-shot interactive renames, just not in portable scripts. We don't include any {{{rename}}} examples here because it's too confusing -- there are two common versions of it and they're totally incompatible with each other.

You can do mass renames with [[BashFAQ/073|Parameter Expansion]], like this:
There are a bunch of different ways to do this, depending on which nonstandard tools you have available. Even with just standard POSIX tools, you can still perform most of the simple cases. We'll show the portable tool examples first.

You can do most non-recursive mass renames with a loop and some [[BashFAQ/073|Parameter Expansions]], like this:
Line 15: Line 13:
{{{
# Rename all "foo" to "bar"
for file in ./*foo*; do mv "$file" "${file//foo/bar}"; done
}}}

{{{
# This removes the extension .zip to all the files.
for file in ./*.zip; do mv "$file" "${file//.zip/ }"; done
}}}

The "--" and "./*" is to protect from problematic filenames that begin with "-".
Here's a similar example, this time replacing spaces in filenames with underscores:
To check what the command would do without actually doing it, you can add an `echo` before the `mv`. This applies to almost(?) every example on this page, so we won't mention it again.

{{{
# POSIX
# This removes the extension .zip from all the files.
for file in ./*.zip; do mv "$file" "${file%.zip}"; done
}}}

The "--" and "./*" are to protect from problematic filenames that begin with "-". You only need one or the other, not both, so pick your favorite.

Here are some similar examples, using Bash-specific parameter expansions:
Line 34: Line 31:
This invokes the external command {{{mv(1)}}} once for each file, so it may not be as efficient as some of the {{{rename}}} implementations.

If you want to do it recursively, then it becomes much more challenging. This example renames {{{*.foo}}} to {{{*.bar}}}:
For more techniques on dealing with files with inconvenient characters in their names, see [[BashFAQ/020|FAQ #20]].

{{{
# Bash
# Replace "foo" with "bar", even if it's not the extension
for file in ./*foo*; do mv "$file" "${file//foo/bar}"; done
}}}

All the above examples invoke the external command `mv(1)` once for each file, so they may not be as efficient as some of the nonstandard implementations.

=== Recursively ===

If you want to rename files recursively, then it becomes much more challenging. This example renames `*.foo` to `*.bar`:
Line 48: Line 55:
{{{
# Bash4 which requires globstar to be enabled. NOT portable!
# Rename all foo files to bar recursively.

shopt -s globstar; for file in /path/to/**/*foo*; do mv -- "${file[@]}" "${file//foo/bar}"; done
}}}
To check what will be the out put of the above command you can add echo before the "mv" so you will get an idea.

For more techniques on dealing with files with inconvenient characters in their names, see [[BashFAQ/020|FAQ #20]].
This example uses Bash 4's `globstar` instead of GNU `find`:

{{{
# Bash 4
# Replace "foo" with "bar" in all files recursively.
# "foo" must NOT appear in a directory name!

shopt -s globstar
for file in /path/to/**/*foo*; do
    mv -- "$file" "${file//foo/bar}"
done
}}}
Line 64: Line 74:
Therefore, any recursive renaming command should only change the filename component of each pathname.  If you need to rename the directories as well, those should be done separately. Furthermore, recursive directory renaming should either be done depth-first (changing only the last component of the directory name in each instance), or in several passes. Depth-first works better in the general case.

Here's an example script that uses depth-first recursion (changes spaces in names to underscores, but you just need to change the ren() function to do anything you want) to rename both files and directories (again, it's easy to modify to make it act only on files or only on directories, or to act only on files with a certain extension, to avoid or force overwriting files, etc.):
Therefore, any recursive renaming command should only change the filename component of each pathname, like this:

{{{
mv "
./FOO/BAR/FILE.TXT" "./FOO/BAR/file.txt"
}}}

If you need to rename the directories as well, those should be done separately. Furthermore, recursive directory renaming should either be done depth-first (changing only the last component of the directory name in each instance), or in several passes. Depth-first works better in the general case.

Here's an example script that uses depth-first recursion (changes spaces in names to underscores, but you just need to change the ren() function to do anything you want) to rename both files and directories. Again, it's easy to modify to make it act only on files or only on directories, or to act only on files with a certain extension, to avoid or force overwriting files, etc.:
Line 73: Line 89:
  [ "$1" != "$newname" ] && mv -- "$1" "$newname"   [[ $1 != "$newname" ]] && mv -- "$1" "$newname"
Line 77: Line 93:
  local i
  cd -- "$1" || exit 1
  for i in *; do
    [ -d "$i" ] && traverse "$i"
    ren "$i"
  local file
  cd -- "$1" || exit
  for file in *; do
    [[ -d $file ]] && traverse "$file"
    ren "$file"
Line 83: Line 99:
  cd .. || exit 1   cd .. || exit
Line 103: Line 119:
To convert filenames to lower case, if you have the utility {{{mmv(1)}}} on your machine, you could simply do:

{{{
#
convert all filenames to lowercase
mmv "*" "#l1"
}}}

Otherwise
, you need something that can take a mixed-case filename as input and give back the lowercase version as output. In Bash 4 and higher, there is a [[BashFAQ/073|parameter expansion]] that can do it:
=== Upper- and lower-case ===

To convert filenames to lower-case with only standard tools, you need something that can take a mixed-case filename as input and give back the lowercase version as output. In Bash 4 and higher, there is a [[BashFAQ/073|parameter expansion]] that can do it:
Line 114: Line 125:
for f in *[[:upper:]]*; do mv -- "$f" "${f,,*}"; done
}}}

Otherwise, {{{tr(1)}}} may be helpful:
for f in *[[:upper:]]*; do mv -- "$f" "${f,,}"; done
}}}

Otherwise, `tr(1)` may be helpful:
Line 122: Line 133:
for file in "$@"
do
for file in "$@"; do
Line 125: Line 135:
    newname=$(echo "$file" | tr '[:upper:]' '[:lower:]') # lower case     newname=$(printf %s "$file" | tr '[:upper:]' '[:lower:]') # lower case
Line 132: Line 142:
We use the fancy range notation, because {{{tr}}} can behave ''very'' strangely when using the {{{A-Z}}} range on some [[locale|locales]]: This example will not handle filenames that end with newlines, because the CommandSubstitution will eat them. The workaround for that is to append a character in the command substitution, and remove it afterward. Thus:
{{{
newname=$(printf %sx "$file" | tr '[:upper:]' '[:lower:]')
newname=${newname%x}
}}}

We use the fancy range notation, because `tr` can behave ''very'' strangely when using the `A-Z` range on some [[locale|locales]]:
Line 139: Line 155:
To make sure you aren't caught by surprise when using {{{tr}}} with ranges, either use the fancy range notations, or set your locale to C. To make sure you aren't caught by surprise when using `tr` with ranges, either use the fancy range notations, or set your locale to C.
Line 154: Line 170:
for file in "$@"
do
for file in "$@"; do
Line 157: Line 172:
    newname=$(echo "$file" | sed 's/[^[:alnum:]_.]/_/g')     newname=$(printf '%s\n' "$file" | sed 's/[^[:alnum:]_.]/_/g')
Line 164: Line 179:
The character class in {{{[]}}} contains all the characters we want to keep (after the {{{^}}}); modify it as needed. The {{{[:alnum:]}}} range stands for all the letters and digits of the current locale.

Here's an example that does the same thing, but this time using Parameter Expansion instead of {{{sed}}}:
The character class in `[]` contains all the characters we want to keep (after the `^`); modify it as needed. The `[:alnum:]` range stands for all the letters and digits of the current locale.

Here's an example that does the same thing, but this time using Parameter Expansion instead of `sed`:
Line 172: Line 187:
   [ -f "$file" ] || continue
   newname=${f//[^[:alnum:]_.]/_}
   [ "$file" = "$newname" ] && continue
   [ -f "$newname" ] && continue
   [[ -f $file ]] || continue
   newname=${file//[![:alnum:]_.]/_}
   [[ $file = "$newname" ]] && continue
   [[ -e $newname ]] && continue
   [[ -L $newname ]] && continue
Line 180: Line 196:
It should be noted that all these examples contain a [[RaceCondition|race condition]] -- an existing file could be overwritten if it is created in between the `[ -f "$newname" ...` and `mv "$file" ...` commands. Solving this issue is beyond the scope of this page, however. It should be noted that all these examples contain a [[RaceCondition|race condition]] -- an existing file could be overwritten if it is created in between the `[ -e "$newname" ...` and `mv "$file" ...` commands. Solving this issue is beyond the scope of this page, however adding the `-i` and (GNU specific) `-T` option to `mv` can reduce its impact.
Line 195: Line 211:
=== Nonstandard tools ===

To convert filenames to lower case, if you have the utility {{{mmv(1)}}} on your machine, you could simply do:

{{{
# convert all filenames to lowercase
mmv "*" "#l1"
}}}

Some GNU/Linux distributions have a {{{rename(1)}}} command; however, the syntax differs from one distribution to the next. Debian uses the [[http://tips.webdesign10.com/files/rename.pl.txt|perl rename script]] (formerly included with Perl; now it is not), which it installs as `prename(1)` and `rename(1)`. Red Hat uses a totally different `rename(1)` command.

The `prename` script is extremely flexible. For example, it can be used to change files to lower-case:

{{{
# convert all filenames to lowercase
prename '$_=lc($_)' ./*
}}}

Alternatively, you can also use:
{{{
# convert all filenames to lowercase
prename 'y/A-Z/a-z/' ./*
}}}

For `prename` to use Unicode instead of ASCII for files encoded in UTF-8:

{{{
# convert all filenames to lowercase using Unicode rules
PERL_UNICODE=SA rename '$_=lc' ./*
}}}

To assume the current [[locale]] charset for filenames:

{{{
rename 'BEGIN{use Encode::Locale qw(decode_argv);decode_argv} $_=lc'
}}}

(note that it still doesn't use the locale's rules for case conversion. For instance, in a Turkish locale, `I` would be converted to `i`, not `ı`).



Or recursively:

{{{
# convert all filenames to lowercase, recursively (assumes a find
# implementation with support for the non-standard -execdir predicate)
#
# Note: this will not change directory names. That's because -execdir
# cd's to the parent directory before running the command. That means
# however that (despite the +), one prename command is executed for
# each file to rename.
find . -type f -name '*[[:upper:]]*' -execdir prename '$_=lc($_)' {} +
}}}

A more efficient and portable approach:

{{{
find . -type f -name '*[[:upper:]]*' -exec prename 's{[^/]*$}{lc($&)}e' {} +
}}}


Or to replace all underscores with spaces:

{{{
prename 's/_/ /g' ./*_*
}}}

How can I rename all my *.foo files to *.bar, or convert spaces to underscores, or convert upper-case file names to lower case?

There are a bunch of different ways to do this, depending on which nonstandard tools you have available. Even with just standard POSIX tools, you can still perform most of the simple cases. We'll show the portable tool examples first.

You can do most non-recursive mass renames with a loop and some Parameter Expansions, like this:

# POSIX
# Rename all *.foo to *.bar
for f in *.foo; do mv -- "$f" "${f%.foo}.bar"; done

To check what the command would do without actually doing it, you can add an echo before the mv. This applies to almost(?) every example on this page, so we won't mention it again.

# POSIX
# This removes the extension .zip from all the files.
for file in ./*.zip; do mv "$file" "${file%.zip}"; done

The "--" and "./*" are to protect from problematic filenames that begin with "-". You only need one or the other, not both, so pick your favorite.

Here are some similar examples, using Bash-specific parameter expansions:

# Bash
# Replace all spaces with underscores
for f in *\ *; do mv -- "$f" "${f// /_}"; done

For more techniques on dealing with files with inconvenient characters in their names, see FAQ #20.

# Bash
# Replace "foo" with "bar", even if it's not the extension
for file in ./*foo*; do mv "$file" "${file//foo/bar}"; done

All the above examples invoke the external command mv(1) once for each file, so they may not be as efficient as some of the nonstandard implementations.

Recursively

If you want to rename files recursively, then it becomes much more challenging. This example renames *.foo to *.bar:

# Bash
# Also requires GNU or BSD find(1)
# Recursively change all *.foo files to *.bar

find . -type f -name '*.foo' -print0 | while IFS= read -r -d '' f; do
  mv -- "$f" "${f%.foo}.bar"
done

This example uses Bash 4's globstar instead of GNU find:

# Bash 4
# Replace "foo" with "bar" in all files recursively.
# "foo" must NOT appear in a directory name!

shopt -s globstar
for file in /path/to/**/*foo*; do
    mv -- "$file" "${file//foo/bar}"
done

The trickiest part of recursive renames is ensuring that you do not change the directory component of a pathname, because something like this is doomed to failure:

mv "./FOO/BAR/FILE.TXT" "./foo/bar/file.txt"

Therefore, any recursive renaming command should only change the filename component of each pathname, like this:

mv "./FOO/BAR/FILE.TXT" "./FOO/BAR/file.txt"

If you need to rename the directories as well, those should be done separately. Furthermore, recursive directory renaming should either be done depth-first (changing only the last component of the directory name in each instance), or in several passes. Depth-first works better in the general case.

Here's an example script that uses depth-first recursion (changes spaces in names to underscores, but you just need to change the ren() function to do anything you want) to rename both files and directories. Again, it's easy to modify to make it act only on files or only on directories, or to act only on files with a certain extension, to avoid or force overwriting files, etc.:

# Bash
ren() {
  local newname
  newname=${1// /_}
  [[ $1 != "$newname" ]] && mv -- "$1" "$newname"
}

traverse() {
  local file
  cd -- "$1" || exit
  for file in *; do
    [[ -d $file ]] && traverse "$file"
    ren "$file"
  done
  cd .. || exit
}

# main program
shopt -s nullglob
traverse /path/to/startdir

Here is another way to recursively rename all directories and files with spaces in their names:

find . -depth -name "* *" -exec bash -c 'dir=${1%/*} base=${1##*/}; mv "$1" "$dir/${base// /_}"' _ {} \;

or, if your version of find accepts it, this is more efficient as it runs one bash for many files instead of one bash per file:

find . -depth -name "* *" -exec bash -c 'for f; do dir=${f%/*} base=${f##*/}; mv "$f" "$dir/${base// /_}"; done' _ {} +

Upper- and lower-case

To convert filenames to lower-case with only standard tools, you need something that can take a mixed-case filename as input and give back the lowercase version as output. In Bash 4 and higher, there is a parameter expansion that can do it:

# Bash 4
for f in *[[:upper:]]*; do mv -- "$f" "${f,,}"; done

Otherwise, tr(1) may be helpful:

# tolower - convert file names to lower case
# POSIX
for file in "$@"; do
    [ -f "$file" ] || continue                # ignore non-existing names
    newname=$(printf %s "$file" | tr '[:upper:]' '[:lower:]')     # lower case
    [ "$file" = "$newname" ] && continue      # nothing to do
    [ -f "$newname" ] && continue             # don't overwrite existing files
    mv -- "$file" "$newname"
done

This example will not handle filenames that end with newlines, because the CommandSubstitution will eat them. The workaround for that is to append a character in the command substitution, and remove it afterward. Thus:

newname=$(printf %sx "$file" | tr '[:upper:]' '[:lower:]')
newname=${newname%x}

We use the fancy range notation, because tr can behave very strangely when using the A-Z range on some locales:

imadev:~$ echo Hello | tr A-Z a-z
hÉMMÓ

To make sure you aren't caught by surprise when using tr with ranges, either use the fancy range notations, or set your locale to C.

imadev:~$ echo Hello | LC_ALL=C tr A-Z a-z
hello
imadev:~$ echo Hello | tr '[:upper:]' '[:lower:]'
hello
# Either way is fine here.

This technique can also be used to replace all unwanted characters in a file name, e.g. with '_' (underscore). The script is the same as above, with only the "newname=..." line changed.

# renamefiles - rename files whose name contain unusual characters
# POSIX
for file in "$@"; do
    [ -f "$file" ] || continue            # ignore non-regular files, etc.
    newname=$(printf '%s\n' "$file" | sed 's/[^[:alnum:]_.]/_/g')
    [ "$file" = "$newname" ] && continue  # nothing to do
    [ -f "$newname" ] && continue         # do not overwrite existing files
    mv -- "$file" "$newname"
done

The character class in [] contains all the characters we want to keep (after the ^); modify it as needed. The [:alnum:] range stands for all the letters and digits of the current locale.

Here's an example that does the same thing, but this time using Parameter Expansion instead of sed:

# renamefiles (more efficient, less portable version)
# Bash
for file in "$@"; do
   [[ -f $file ]] || continue
   newname=${file//[![:alnum:]_.]/_}
   [[ $file = "$newname" ]] && continue
   [[ -e $newname ]] && continue
   [[ -L $newname ]] && continue
   mv -- "$file" "$newname"
done

It should be noted that all these examples contain a race condition -- an existing file could be overwritten if it is created in between the [ -e "$newname" ... and mv "$file" ... commands. Solving this issue is beyond the scope of this page, however adding the -i and (GNU specific) -T option to mv can reduce its impact.

One final note about changing the case of filenames: when using GNU mv, on many file systems, attempting to rename a file to its lowercase or uppercase equivalent will fail. (This applies to Cygwin on DOS/Windows systems using FAT or NTFS file systems; to GNU mv on Mac OS X systems using HFS+ in case-insensitive mode; as well as to Linux systems which have mounted Windows/Mac file systems, and possibly many other setups.) GNU mv checks both the target names before attempting a rename, and due to the file system's mapping, it thinks that the destination "already exists":

mv README Readme    # fails with GNU mv on FAT file systems, etc.

The workaround for this is to rename the file twice: first to a temporary name which is completely different from the original name, then to the desired name.

mv README tempfilename &&
mv tempfilename Readme

Nonstandard tools

To convert filenames to lower case, if you have the utility mmv(1) on your machine, you could simply do:

# convert all filenames to lowercase
mmv "*" "#l1"

Some GNU/Linux distributions have a rename(1) command; however, the syntax differs from one distribution to the next. Debian uses the perl rename script (formerly included with Perl; now it is not), which it installs as prename(1) and rename(1). Red Hat uses a totally different rename(1) command.

The prename script is extremely flexible. For example, it can be used to change files to lower-case:

# convert all filenames to lowercase
prename '$_=lc($_)' ./*

Alternatively, you can also use:

# convert all filenames to lowercase
prename 'y/A-Z/a-z/' ./*

For prename to use Unicode instead of ASCII for files encoded in UTF-8:

# convert all filenames to lowercase using Unicode rules
PERL_UNICODE=SA rename '$_=lc' ./*

To assume the current locale charset for filenames:

rename 'BEGIN{use Encode::Locale qw(decode_argv);decode_argv} $_=lc'

(note that it still doesn't use the locale's rules for case conversion. For instance, in a Turkish locale, I would be converted to i, not ı).

Or recursively:

# convert all filenames to lowercase, recursively (assumes a find
# implementation with support for the non-standard -execdir predicate)
#
# Note: this will not change directory names. That's because -execdir
# cd's to the parent directory before running the command. That means
# however that (despite the +), one prename command is executed for
# each file to rename.
find . -type f -name '*[[:upper:]]*' -execdir prename '$_=lc($_)' {} +

A more efficient and portable approach:

find . -type f -name '*[[:upper:]]*' -exec prename 's{[^/]*$}{lc($&)}e' {} +

Or to replace all underscores with spaces:

prename 's/_/ /g' ./*_*


CategoryShell

BashFAQ/030 (last edited 2020-06-10 14:10:22 by GreyCat)