|
Size: 12909
Comment: prev/next links
|
Size: 5
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| [[BashGuide/Patterns|<- Patterns]] | [[BashGuide/TestsAndConditionals|Tests and Conditionals ->]] ---- == Arrays == As mentioned earlier, [[BASH]] provides three types of parameters: Strings, Integers and Arrays. Strings are without a doubt the most used parameter type. But they are also the most misused parameter type. It is important to remember that a string holds just '''one''' element. Capturing the output of a command, for instance, and putting it in a string parameter means that parameter holds just '''one''' string of characters, regardless of whether that string represents twenty filenames, twenty numbers or twenty names of people. And as is always the case when you put multiple items in a single string, these multiple items must be somehow delimited from each other. We, as humans, can usually decipher what the different filenames are when looking at a string. We assume that, perhaps, each line in the string represents a filename, or each word represents a filename. While this assumption is understandable, it is also inherently flawed. Each single filename can contain every character you might want to use to separate the filenames from each other in a string. That means there's technically no telling where the first filename in the string ends, because there's no character that can say: "I denote the end of this filename" because that character itself could be part of the filename. Often, people make this mistake: {{{ $ files=$(ls); cp $files /backups/ }}} This attempt at backing up our files in the current directory is flawed. We put the output of `ls` in a string called `files` and then use the '''unquoted''' `$files` parameter expansion to cut that string into arguments (relying on ''Word Splitting''). As mentioned before, argument and word splitting cuts a string into pieces wherever there is whitespace. Relying on it means we assume that none of our filenames will contain any whitespace. If they do, the filename will be cut in half or more. Conclusion: '''bad'''. The only safe way to represent '''multiple''' string elements in bash is through the use of arrays. Arrays are types that '''map integers to strings'''. That basically means that they hold a numbered list of strings. Since each of these strings is a separate entity (element), it can safely contain any character. For the best results and the least headaches, remember that if you have a list of things, you should always put it in an array. (Unlike some other programming languages, bash does not offer lists, tuples, etc. Just arrays, and associative arrays (which are new in bash 4). We do not cover associative arrays here yet.) -------- . ''Array'': An array is a numbered list of strings: It maps integers to strings. -------- <<Anchor(Creating_Arrays)>> === Creating Arrays === There are several ways you can create or fill your array with data. There is no one single true way: the method you'll need depends on where your data comes from and what it is. The easiest way to create a simple array with data is by using the `=()` syntax: {{{ $ names=("Bob" "Peter" "$USER" "Big Bad John") }}} This syntax is great for creating arrays with static data or a known set of string parameters, but it gives us very little flexibility for adding lots of array elements. If you need more flexibility, you can also specify explicit indexes: {{{ $ names=([0]="Bob" [1]="Peter" [20]="$USER" [21]="Big Bad John") # or... $ names[0]="Bob" }}} Notice that there is a gap between indices 1 and 20 in this example. An array with holes in it is called a ''sparse array''. Bash allows this, and it can often be quite useful. If you want to fill an array with filenames, then you'll probably want to use `Globs` in there: {{{ $ photos=(~/"My Photos"/*.jpg) }}} Notice here that we quoted the `My Photos` part because it contains a space. If we hadn't quoted it, bash would have split it up into `photos=('~/My' 'Photos/'*.jpg )` which is obviously '''not''' what we want. Also notice that we quoted '''only''' the part that contained the space. That's because we cannot quote the `~` or the `*`; if we do, they'll become literal and bash won't treat them as special characters anymore. Creating arrays with a bunch of filenames becomes really easy like this. So remember to '''never''' use `ls`: {{{ $ files=$(ls) # BAD, BAD, BAD! $ files=(*) # Good! }}} The first would create a '''string''' with the output of `ls`. That string cannot possibly be used safely for reasons mentioned in the `Arrays` introduction. The second statement gives us an array where each filename is a separate element. Perfect. Now, sometimes we want to build an array from a string or the output of a command. Commands (generally) just output strings: for instance, running a `find` command will enumerate filenames, and separate these filenames with newlines (putting each filename on a separate line). So to parse that one big string into an array we need to tell bash what character delimits the parts of the string that we want to put in separate array elements. That's what `IFS` is used for: {{{ $ IFS=. read -a ip_elements <<< "127.0.0.1" }}} Here we use `IFS` with the value `.` to cut the given ip address into array elements wherever there's a `.`, resulting in an array with the elements `127`, `0`, `0` and `1`. Very often, though, it's impossible to safely tell what the delimiter is. For instance, when `find` outputs filenames separated by a newline, using a newline to delimit our parsing of that output string is flawed in that a filename can itself contain a newline. We cannot assume that each newline means: "A new filename follows", which means we cannot parse the output safely. And if we can't even parse it safely, there's little point in parsing it badly and then putting it in a safe array: The array would just contain badly parsed data. The answer to this problem is `NULL` bytes. The main difference between strings and the output of commands is that the latter is a `stream`, not a string. Streams are just like strings with one big difference: they can contain `NULL` bytes, while strings cannot. A `NULL` byte is a byte which is just all zeros: `00000000`. The reason that they can't be used in strings is an artifact of the `C` programming language: `NULL` bytes are used in C to mark the end of a string. Since bash is written in C and uses C's native strings, it inherits that behavior. Streams '''can''' contain `NULL` bytes, and we will use them to delimit our data. Filenames cannot contain `NULL` bytes (since they're implemented as C strings by Unix), and neither can the vast majority of things we would want to store in a program (people's names, IP addresses, etc.). That makes `NULL` a great candidate for separating elements in a stream. Usually, the command you want to read the output of has an option that makes it output its data separated by `NULL` bytes rather than newlines or something else. `find` has the option `-print0`, which we'll use in this example: {{{ files=() while read -r -d $'\0'; do files+=("$REPLY") done < <(find /foo -print0) }}} This is the only safe way of parsing a command's output into a string. Understandably, it looks a little confusing and convoluted at first. So let's take it apart: The first line `files=()` creates an empty array named `files`. We're using a `while` loop that runs a `read` command each time. The `read` command uses the `-d $'\0'` option, which means that instead of reading a line (up to a newline), we're reading up to a `NULL` byte (`\0`). Once `read` has read some data and encountered a `NULL` byte, the `while` loop's body is executed. We put what we read (which is in the parameter `REPLY`) into our array. To do this, we use the `+=()` syntax. This syntax adds an (or more) element(s) to the end of our array. Finally, the `< <(..)` syntax is a combination of ''File Redirection'' (`<`) and ''Process Substitution'' (`<(..)`) which is used to redirect the output of the `find` command into our `while` loop. The `find` command itself uses the `-print0` option as mentioned before to tell it to separate the filenames it finds with a `NULL` byte. -------- . '''Good Practice: <<BR>> Arrays are a safe list of strings. They are perfect for representing multiple filenames.<<BR>> Make sure to always use the `NULL`-byte method of parsing command output whenever you're parsing output that can contain any string character within an element.''' ---- . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Arrays|Arrays]]''' ---- . '''In the FAQ: <<BR>> [[BashFAQ/005|How can I use array variables?]]<<BR>> [[BashFAQ/006|How can I use variable variables (indirect variables, pointers, references) or associative arrays?]]''' -------- <<Anchor(Using_Arrays)>> === Using Arrays === Walking over array elements is really easy. Because an array is such a safe medium of storage, we can simply use a `for` loop to iterate over its elements: {{{ $ for file in "${myfiles[@]}"; do > cp "$file" /backups/ > done }}} Notice the syntax used to '''expand''' the array here. We use the '''quoted''' form: `"${arrayname[@]}"`. That causes bash to replace it with every single element in the array, properly quoted. For instance, these are identical: {{{ $ names=("Bob" "Peter" "$USER" "Big Bad John") $ for name in "${names[@]}"; do :; done $ for name in "Bob" "Peter" "$USER" "Big Bad John"; do :; done }}} Remember to '''quote''' the `${arrayname[@]}` properly. If you don't you lose all benefit of using an array at all: you're telling bash it's OK to wordsplit your array elements to pieces and break everything again. Another use of `"${arrayname[@]}"` is simplifying the above loop, for instance: {{{ cp -- "${myfiles[@]}" /backups/ }}} This runs the `cp` command, replaces the `"${myfiles[@]}"` part by each filename in the `myfiles` array, properly quoted, causing `cp` to safely copy them to your backups. (We use `--` to tell `cp` that there are no options after that point. This is just in case one of our filenames begins with a `-` character, which could confuse `cp` into thinking it is an option.) You can also expand single array elements by referencing their element number (called '''index'''). Though remember that arrays are ''zero-based'', which means that their '''first element''' has the index '''zero''': {{{ $ echo "The first name is: ${names[0]}" $ echo "The second name is: ${names[1]}" }}} There is also a second form of expanding all array elements, which is `"${arrayname[*]}"`. This form is '''ONLY''' useful for converting arrays back into a single string. The main purpose for this is outputting the array to humans: {{{ $ names=("Bob" "Peter" "$USER" "Big Bad John") $ echo "Today's contestants are: ${names[*]}" Today's contestants are: Bob Peter lhunath Big Bad John }}} Remember to still keep everything nicely '''quoted'''! If you don't keep `${arrayname[*]}` quoted, once again bash's ''Wordsplitting'' will cut it into bits. You can combine `IFS` with `"${arrayname[*]}"` to indicate the character to use to delimit your array elements as you merge them into a single string. This is handy, for example, when you want to comma delimit names: {{{ $ names=("Bob" "Peter" "$USER" "Big Bad John") $ ( IFS=,; echo "Today's contestants are: ${names[*]}" ) Today's contestants are: Bob,Peter,lhunath,Big Bad John }}} Notice how in this example we put the `IFS=,; echo ...` statement in a ''Subshell'' by wrapping `(` and `)` around it. We do this because we don't want to change the default value of `IFS` in the main shell. As soon as the subshell exits, `IFS` is still its default value and no longer just a comma. This is important because `IFS` is used for a lot of things, and changing its value to something non-default will result in very odd behavior if you don't expect it! Alas, the `"${array[*]}"` expansion only uses the ''first'' character of `IFS` to join the elements together. If we wanted to separate the names in the previous example with a comma and a space, we would have to use some other technique (for example, a `for` loop). -------- . '''Good Practice: <<BR>> Always quote your array expansions properly, just like you would your normal parameter expansions.<<BR>>Use `"${myarray[@]}"` to expand all your array elements and ONLY use `"${myarray[*]}"` when you want to merge all your array elements into a single string.''' -------- [[BashGuide/Patterns|<- Patterns]] | [[BashGuide/TestsAndConditionals|Tests and Conditionals ->]] |
zas |
zas