Differences between revisions 21 and 24 (spanning 3 versions)
Revision 21 as of 2012-02-01 15:38:16
Size: 4435
Editor: GreyCat
Comment: it's a useful trick for bash < 4 (I'll add a warning)
Revision 24 as of 2014-05-22 02:20:17
Size: 6154
Editor: ormaaj
Comment: Work on examples
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
Bash version 4 allows zero-padding and ranges in its BraceExpansion:

{{{
    # Bash 4 / zsh
    for i in {01..10}; do
        ...
}}}

All of the other solutions on this page will assume Bash earlier than 4.0, or a non-Bash shell.
Line 7: Line 17:
    # Bash     # Bash / ksh / zsh
Line 10: Line 20:
        echo $i         echo "$i"
Line 14: Line 24:
Output:
{{{
01
   02
   03
   [...]
}}}

T
he same thing can be accomplished more concisely like this:
In Bash 3, you can use ranges inside brace expansion (but not zero-padding). Thus, the same thing can be accomplished more concisely like this:
Line 25: Line 27:
    # Bash     # Bash 3
Line 28: Line 30:
        echo $i         echo "$i"
Line 35: Line 37:
    # Bash     # Bash 3
Line 38: Line 40:
        echo $i         echo "$i"
Line 46: Line 48:
Some may prefer the following Q&D solution (producing "001" through "015"):
This gets tedious for large sequences, but there are other ways, too. If you have the {{{printf}}} command (which is a Bash builtin, and is also POSIX standard), it can be used to format a number:
Line 49: Line 50:
    # Bash
    for i in {1000..1015}
   do
      i=${i:1}
      echo $i
    # Bash / ksh93 / zsh
    for ((i = 1; i <= 10; i++)); do
        i=$(printf %02d "$i")
        ...
Line 57: Line 57:

This gets tedious for large sequences, but there are other ways, too. If you have the {{{printf}}} command (which is a Bash builtin, and is also POSIX standard), it can be used to format a number:
{{{
    # Bash
    for ((i=1; i<=10; i++))
    do
        printf "%02d " "$i"
    done
}}}

In Bash 3, you can use ranges inside brace expansion. Also, since {{{printf}}} will implicitly loop if given more arguments than format specifiers, you can simplify this enormously:
Also, unlike the C library `printf`, since {{{printf}}} will implicitly loop if given more arguments than format specifiers, you can simplify this enormously:
Line 70: Line 60:
   printf "%03d\n" {1..300}    printf '%03d\n' {1..300}
Line 76: Line 66:
   # START and END are variables containing integers
   eval printf '"%03d\n"' {$START..$END}
   # start and end are variables containing integers
   eval "printf '%03d\n' {$start..$end}"
Line 80: Line 70:
The `eval` is needed here because you cannot have variables in a brace expansion -- only constants. The extra quotes are required by the `eval` so that our `\n` isn't changed to an `n`. Given how messy that `eval` solution is, please give serious thought to using the `for` loop instead. The `eval` is required in Bash because for each command it performs an initial pass of evaluation, going through each word to process brace expansions prior to any other evaluation step. The traditional Csh implementation, which all other applicable shells follow, insert the brace expansion pass sometime between the processing of other expansions and pathname expansion, thus parameter expansion has already been performed by the time words are scanned for brace expansion. There are various pros and cons to Bash's implementation, this being probably the most frequently cited drawback. Given how messy that `eval` solution is, please give serious thought to using a `for` or `while` loop with shell arithmetic instead.
Line 82: Line 72:
Also, bash version 4 allows zero-padding in its brace expansion: The ksh93 method for specifying field width for sequence expansion is to add a (limited) `printf` format string to the syntax, which is used to format each expanded word. This is somewhat more powerful, but unfortunately incompatible with bash, and ksh does not understand Bash's field padding scheme:
Line 85: Line 75:
    # Bash 4
    echo {01..10}

    for i in {01
..10}; do ...
    #ksh93
    echo {0..10..2%02d}
Line 91: Line 79:
The KornShell has the {{{typeset}}} command to specify the number of leading zeros: ksh93 also has a variable attribute that specifies a field with to pad with leading zeros whenever the variable is referenced. The concept is similar to other attributes supported by Bash such as case modification. Note that ksh never interprets octal literals.
Line 93: Line 82:
    # Korn     # ksh93 / mksh / zsh
Line 115: Line 104:
(That may be helpful if your version of {{{seq(1)}}} lacks {{{printf}}}-style format specifiers. Since it's a nonstandard external tool, it's good to keep your options open.) (That may be helpful if you are not using Bash, but you have `seq(1)`, and your version of {{{seq(1)}}} lacks {{{printf}}}-style format specifiers. That's a pretty odd set of restrictions, but I suppose it's theoretically possible. Since `seq` is a nonstandard external tool, it's good to keep your options open.)
Line 117: Line 106:
Be warned however that `seq` might be considered bad style; it's even mentioned in [[BashGuide/Practices#Don.27t_Ever_Do_These|Don't Ever Do These]]. Be warned however that using `seq` might be considered bad style; it's even mentioned in [[BashGuide/Practices#Don.27t_Ever_Do_These|Don't Ever Do These]].

Some BSD-derived systems have `jot(1)` instead of `seq(1)`. In accordance with the glorious tradition of Unix, it has a completely incompatible syntax:
{{{
   # POSIX shell, OpenBSD et al.
   printf "%02d\n" $(jot 10 1)

   # Bourne shell, OpenBSD (at least)
   jot -w %02d 10 1
}}}
Line 136: Line 134:
   # COUNT variable contains an integer
   awk "BEGIN {for (i=1;i<$COUNT;i++) {printf(\"%03d\n\",i)} }"
   # count variable contains an integer
   awk -v count="$count" 'BEGIN {for (i=1;i<=count;i++) {printf("%03d\n",i)} }'

   # Bourne, with Solaris's decrepit and useless awk:
   awk "BEGIN {for (i=1;i<=$count;i++) {printf(\"%03d\\n\",i)} }"
Line 139: Line 140:

----
Line 152: Line 155:
      # other commands       sleep 5
   done
}}}

Or, avoiding the subshells (requires bash 3.1):
{{{
   # Bash 3.1
   for i in {1..100}; do
      printf -v n %03d $i
      wget "$prefix$n.jpg"
      sleep 5

How can I use numbers with leading zeros in a loop, e.g. 01, 02?

As always, there are different ways to solve the problem, each with its own advantages and disadvantages.

Bash version 4 allows zero-padding and ranges in its BraceExpansion:

    # Bash 4 / zsh
    for i in {01..10}; do
        ...

All of the other solutions on this page will assume Bash earlier than 4.0, or a non-Bash shell.

If there are not many numbers, BraceExpansion can be used:

    # Bash / ksh / zsh
    for i in 0{1,2,3,4,5,6,7,8,9} 10
    do
        echo "$i"
    done

In Bash 3, you can use ranges inside brace expansion (but not zero-padding). Thus, the same thing can be accomplished more concisely like this:

    # Bash 3
    for i in 0{1..9} 10
    do
        echo "$i"
    done

Another example, for output of 0000 to 0034:

    # Bash 3
    for i in {000{0..9},00{10..34}}
    do
        echo "$i"
    done

    # using the outer brace instead of just adding them one next to the other
    # allows to use the expansion, for instance, like this:
    wget 'http://foo.com/adir/thepages'{000{0..9},00{10..34}}'.html'

This gets tedious for large sequences, but there are other ways, too. If you have the printf command (which is a Bash builtin, and is also POSIX standard), it can be used to format a number:

    # Bash / ksh93 / zsh
    for ((i = 1; i <= 10; i++)); do
        i=$(printf %02d "$i")
        ...
    done

Also, unlike the C library printf, since printf will implicitly loop if given more arguments than format specifiers, you can simplify this enormously:

   # Bash 3
   printf '%03d\n' {1..300}

If you don't know in advance what the starting and ending values are:

   # Bash 3
   # start and end are variables containing integers
   eval "printf '%03d\n' {$start..$end}"

The eval is required in Bash because for each command it performs an initial pass of evaluation, going through each word to process brace expansions prior to any other evaluation step. The traditional Csh implementation, which all other applicable shells follow, insert the brace expansion pass sometime between the processing of other expansions and pathname expansion, thus parameter expansion has already been performed by the time words are scanned for brace expansion. There are various pros and cons to Bash's implementation, this being probably the most frequently cited drawback. Given how messy that eval solution is, please give serious thought to using a for or while loop with shell arithmetic instead.

The ksh93 method for specifying field width for sequence expansion is to add a (limited) printf format string to the syntax, which is used to format each expanded word. This is somewhat more powerful, but unfortunately incompatible with bash, and ksh does not understand Bash's field padding scheme:

    #ksh93
    echo {0..10..2%02d}

ksh93 also has a variable attribute that specifies a field with to pad with leading zeros whenever the variable is referenced. The concept is similar to other attributes supported by Bash such as case modification. Note that ksh never interprets octal literals.

    # ksh93 / mksh / zsh
    $ typeset -Z3 i=4
    $ echo $i
    004

If the command seq(1) is available (it's part of GNU sh-utils/coreutils), you can use it as follows:

    seq -w 1 10

or, for arbitrary numbers of leading zeros (here: 3):

    seq -f "%03g" 1 10

Combining printf with seq(1), you can do things like this:

   # POSIX shell, GNU utilities
   printf "%03d\n" $(seq 300)

(That may be helpful if you are not using Bash, but you have seq(1), and your version of seq(1) lacks printf-style format specifiers. That's a pretty odd set of restrictions, but I suppose it's theoretically possible. Since seq is a nonstandard external tool, it's good to keep your options open.)

Be warned however that using seq might be considered bad style; it's even mentioned in Don't Ever Do These.

Some BSD-derived systems have jot(1) instead of seq(1). In accordance with the glorious tradition of Unix, it has a completely incompatible syntax:

   # POSIX shell, OpenBSD et al.
   printf "%02d\n" $(jot 10 1)

   # Bourne shell, OpenBSD (at least)
   jot -w %02d 10 1

Finally, the following example works with any BourneShell derived shell (which also has expr and sed) to zero-pad each line to three bytes:

   # Bourne
   i=0
   while test $i -le 10
   do
       echo "00$i"
       i=`expr $i + 1`
   done |
       sed 's/.*\(...\)$/\1/g'

In this example, the number of '.' inside the parentheses in the sed command determines how many total bytes from the echo command (at the end of each line) will be kept and printed.

But if you're going to rely on an external Unix command, you might as well just do the whole thing in awk in the first place:

   # Bourne
   # count variable contains an integer
   awk -v count="$count" 'BEGIN {for (i=1;i<=count;i++) {printf("%03d\n",i)} }'

   # Bourne, with Solaris's decrepit and useless awk:
   awk "BEGIN {for (i=1;i<=$count;i++) {printf(\"%03d\\n\",i)} }"


Now, since the number one reason this question is asked is for downloading images in bulk, you can use the examples above with xargs(1) and wget(1) to fetch files:

   almost any example above | xargs -i% wget $LOCATION/%

The xargs -i% will read a line of input at a time, and replace the % at the end of the command with the input.

Or, a simpler example using a for loop:

   # Bash 3
   for i in {1..100}; do
      wget "$prefix$(printf %03d $i).jpg"
      sleep 5
   done

Or, avoiding the subshells (requires bash 3.1):

   # Bash 3.1
   for i in {1..100}; do
      printf -v n %03d $i
      wget "$prefix$n.jpg"
      sleep 5
   done


CategoryShell

BashFAQ/018 (last edited 2019-08-21 16:24:29 by GreyCat)