9273
Comment: more on bash's recursive evaluation

← Revision 33 as of 20190617 15:33:12 ⇥
10978
10# fails with negative numbers, because bash hates you and wants your world to burn

Deletions are marked like this.  Additions are marked like this. 
Line 133:  Line 133: 
/!\ The `10#` prefix trick only works with signless numbers; see warnings below. 

Line 192:  Line 194: 
=== Pitfall: Base prefix with signed numbers === The ''base''`#` prefix feature only works with signless numbers. It does ''not'' work with negative numbers (because they necessarily have a `` character), nor with numbers that have a leading `+`. Because of how bash parses numbers, `10#1` is interpreted as three separate tokens: `10#`, ``, `1`. The `10#` has an empty numeric part to the right of the `#`, so it gets parsed as zero. `10#1` is therefore equivalent to `01` which is obviously not what was intended. This is a subtle problem waiting to happen if you use an expression like `$(( 10 * 10#$j ))` where `j` could potentially be negative. If `j` is 5, the expression is parsed as 10 * 0  5, which has a final value of 5, rather than 50 as you might have expected. It also fails to perform the base conversion of a negative number. If `i` is `019`, then `$((10#$i))` is parsed as `$((0019))` and gives the octal error ("value too great for base"). The following workaround has [[https://lists.gnu.org/archive/html/bugbash/201807/msg00033.htmlbeen suggested]] for handling base conversion of potentially negative or `+`prefixed numbers: {{{ i=$(( ${i%%[!+]*}10#${i#[+]} )) }}} This splits the number into its sign and digit pieces, and moves the sign in front of the `10#`, which gives the desired result. There are a great many shell script applications where negative inputs will simply never happen (e.g. `date +%d` will never give a negative value). So, it's quite possible to write most scripts without worrying about this. But you should keep it in mind. 
Arithmetic in BASH is integer math only. You can't do floating point math in Bash; if you need that capability, see Bash FAQ #22.
The $[ ] syntax is deprecated. Please use $(( )) instead.
Arithmetic Expansion
POSIX sh (and all shells based on it, including Bash and ksh) uses the $(( )) syntax to do arithmetic, using the same syntax as C. (See the Bash hackers article for the full syntax.) Bash calls this an "Arithmetic Expansion", and it obeys the same basic rules as all other $... substitutions.
$(( )) is the first example of a math context, meaning a context where the syntax and semantics of C's integer arithmetic are used. This will be discussed in more detail below.
Here are a few examples using $(( )):
# POSIX sh i=$((j + 3)) lvcreate L "$((24 * 1024))" n lv99 vg99 q=$((29 / 6)) r=$((29 % 6)) if test "$((a%4))" = 0; then ...
Notes:
Arithmetic operators include all of the C operators (arithmetic, bit shifting/masking, ternary ?:), plus ** for exponentiation.
You may use commas to separate multiple expressions within a single math context. The final value of the arithmetic expression is that of the last commadelimited expression.
In bash, all arithmetic is done with signed integers using C's intmax_t variable type (typically 64 bits, but platformdependent). Before bash 2.05b, it used long int variables (typically 32 bits).
 Variable names in a math context are substituted with their values (unset or empty variables are evaluated as 0).
 In POSIX sh, the substituted value must be an integer.
 In bash, the substituted value is recursively evaluated as a new arithmetic expression.
 Numbers without leading 0 are treated as base 10. Numbers with a leading 0x are treated as base 16. Numbers with a leading 0 (not followed by x) are treated as base 8.
In bash, you may put 10# or 16# (etc.) in front of a number to force it to be interpreted in a given base  more on this later.
Arithmetic Commands
Bash also offers two forms of commands that use a math context. The expressions within the math context follow the same rules as the expressions inside $(( )), but since we have a command, we also get an exit status, and side effects.
The exit status is based on the value of the (last) expression. If the expression evaluates to 0, the command is considered "false" and returns 1. Otherwise, the command is considered "true" and returns 0. See below for examples of this.
The first arithmetic command is let:
let a=17+23 echo "a = $a" # Prints a = 40
Note that each arithmetic expression has to be passed as a single argument to the let command, so you need quotes if there are spaces or globbing characters. Thus:
let a=17 + 23 # WRONG let a="17 + 23" # Right let 'a = 17 + 23' # Right let a=17 a+=23 # Right (2 arithmetic expressions) let a[1]=1+1 # Wrong (try after touch a1=1+1 or with shopt s failglob) let 'a[1]=1+1' # Right let a\[1\]=1+1 # Right, but very odd.
The second command is (( )). It works identically to let, but you don't need to quote the expressions because they are delimited by the (( and )) syntax. For example:
((a=$a+7)) # Add 7 to a ((a = a + 7)) # Add 7 to a. Identical to the previous command. ((a += 7)) # Add 7 to a. Identical to the previous command. ((a = RANDOM % 10 + 1)) # Choose a random number from 1 to 10. # % is modulus, as in C.
> or < inside (( )) means greater/less than, not output/input redirection. An expression containing one of these evalutes to either 1 (if the comparison is true) or 0 (if false).
if ((a > 5)); then echo "a is more than 5"; fi
(( )) is used more widely than let, because it fits so well into an if or while command.
Finally, a note on the exit status of commands, and the notions of "true" and "false", is in order. When Bash runs a command, that command will return an exit status from 0 to 255. 0 is considered "success" (which is "true" when used in the context of an if or while command). However, when evaluating an arithmetic expression, C language rules (0 is false, anything else is true) apply.
Some examples:
true; echo "$?" # Writes 0, because a successful command returns 0. ((10 > 6)); echo "$?" # Also 0. An arithmetic command returns 0 for true. echo "$((10 > 6))" # Writes 1. An arithmetic expression evaluates to 1 for true.
In addition to a comparison returning 1 for true, an arithmetic command that evaluates to any nonzero value returns true as an exit status.
if ((1)); then echo true; fi # Writes true.
This also lets you use "flag" variables, just like in a C program:
found=0 while ...; do ... if something; then found=1; fi # Found one! Keep going. ... done if ((found)); then ...
This also means that every arithmetic command we run is returning an exit status. Most scripts ignore these, but if you are using set e you may be unpleasantly surprised when your program aborts because you assigned 0 to a number.
Math Contexts
As we've seen, the insides of $(( )) and (( )), and the arguments of let, are math contexts.
Also, (nonassociative) array indices are a math context:
n=0 while read line; do array[n++]=$line # array[] forces a numeric context done
In case it wasn't obvious, the (( )) in a Cstyle for command are a math context. Or three separate math contexts, depending on your point of view.
for ((i=0, j=0; i<100; i++, j+=5)); do ...
Finally, the start and length arguments of Bash's ${var:start:length} parameter expansion are math contexts.
echo "${string:i+2:len2}"
Leading Zeros and Base Selection
The 10# prefix trick only works with signless numbers; see warnings below.
There is one common pitfall with arithmetic expressions in Bash: numbers with leading zeroes are treated as octal. For example,
# Suppose today is September 19th. month=$(date +%m) next_month=$(( (month == 12) ? 1 : month+1 )) # bash: 09: value too great for base (error token is "09")
This causes great confusion among people who are extracting zeropadded numbers from various sources (dates and times are by far the most common) and then doing math on them without sanitizing them first. It's especially bad if you write a program like this in March, test it, roll it out... and then it doesn't blow up until August 1.
There are two possible solutions. The first is, obviously, to remove the leading zeroes from the numbers before doing math with them. This can be done in several ways. First, a simple loop:
# This removes leading zeroes from a, one at a time. while [[ $a = 0* ]]; do a=${a#0}; done
You can also do it with extended globs; see FAQ #67 for more information. Or, you could use sed; that may be more efficient if you're reading many numbers from a stream, and can arrange to sanitize them all in one command, rather than one by one.
With an extended glob:
shopt s extglob # This removes leading zeroes from a, all at once. a=${a##+(0)}
Finally, Bash can do it with an arithmetic expression:
month=$(date +%m) month=$((10#$month)) # Strip leading zeros by forcing evaluation in base 10
The second solution is to force Bash to treat all numbers as base 10 by prefixing them with 10# every time they are used.
a=008 let b=a+1 # Generates an error because 008 is not valid in octal. let b=10#$a+1 # Force a to be treated as base 10. Note: the $ is required.
Generally, it's better to strip the zeroes once than to keep doing 10# on the same value repeatedly.
Here is a function to convert numbers in other bases to decimal (base 10):
frombase() { echo "$(( $1#$2 ))" } # Examples: frombase 16 ffe # prints 4094 frombase 2 100100 # prints 36
Pitfall: Base prefix with signed numbers
The base# prefix feature only works with signless numbers. It does not work with negative numbers (because they necessarily have a  character), nor with numbers that have a leading +. Because of how bash parses numbers, 10#1 is interpreted as three separate tokens: 10#, , 1. The 10# has an empty numeric part to the right of the #, so it gets parsed as zero. 10#1 is therefore equivalent to 01 which is obviously not what was intended.
This is a subtle problem waiting to happen if you use an expression like $(( 10 * 10#$j )) where j could potentially be negative. If j is 5, the expression is parsed as 10 * 0  5, which has a final value of 5, rather than 50 as you might have expected.
It also fails to perform the base conversion of a negative number. If i is 019, then $((10#$i)) is parsed as $((0019)) and gives the octal error ("value too great for base").
The following workaround has been suggested for handling base conversion of potentially negative or +prefixed numbers:
i=$(( ${i%%[!+]*}10#${i#[+]} ))
This splits the number into its sign and digit pieces, and moves the sign in front of the 10#, which gives the desired result.
There are a great many shell script applications where negative inputs will simply never happen (e.g. date +%d will never give a negative value). So, it's quite possible to write most scripts without worrying about this. But you should keep it in mind.
Integer Declaration
Integer declaration is considered harmful. Don't do this. It makes your code difficult to read, because you need to know whether a given variable is declared as an integer or not, to know what a command does.
Variables may be declared as integers so that any subsequent assignments to them will always assume a numeric context. Essentially any variable that's declared as an integer acts as if you had a let command in front of it when you assign to it. For example:
unset b # Forget any previous declarations b=7+5; echo "$b" # Prints 7+5 declare i b # Declare b as an integer b=7+5; echo "$b" # Prints 12
Also note that regardless of whether you use declare i, bash does not store numbers in binary form. They are still stored as strings, with string/integer conversions happening on the fly every time arithmetic is done.