<- Data structures | Example 0: Filtering a list | Example 1: Modifying a config file ->

Example 0: Filtering a list

This example comes from a #bash user. I consider this question easy, especially if the output is only intended for human convenience. (See the Implementation section to understand this caveat.)

Specification

<user> let's say I have a list like the output of grep where the left
       column is the same item repeated a lot
<user> how can I grab the line that is just where the left column changes
<user> i.e. what you might get by specifying -m 1 to grep

Discussion

The two direct references to grep as well the mention of the GNU grep option -m 1 tell us that the user is, most likely, actually filtering grep output. So let's start by taking a look at that.

hobbit:~$ grep PATH /etc/skel/.*
/etc/skel/.kshrc:FPATH=/usr/share/ksh/functions:~/.func
/etc/skel/.profile:# set PATH so it includes user's private bin if it exists
/etc/skel/.profile:    PATH="$HOME/bin:$PATH"
/etc/skel/.profile:# set PATH so it includes user's private bin if it exists
/etc/skel/.profile:    PATH="$HOME/.local/bin:$PATH"

When grep is given multiple filename arguments, it precedes each output line with the filename from which the line came, and a colon. What the user wants is to see only the first matching line from each file.

Implementation

We only need to read the grep input line by line, and keep track of the previous line's filename. If the current line's filename is different from the previous one, then we print the current line. So, we just need one variable to track the previous filename, and a while read loop.

# First implementation: almost correct.
filter() {
    local prev="" file line
    while IFS=: read -r file line; do
        if [[ "$file" != "$prev" ]]; then
            printf '%s:%s\n' "$file" "$line"
        fi
        prev=$file
    done
}

This implementation uses IFS=: read -r file line to split each line of input into a filename field, and a "rest of the line" field. There is actually a pitfall associated with this solution: the line variable will not contain the full text of the input line if the matched line happens to end with a colon, and there are no other colons in the matched line. This is rare, but possible. We can apply the pitfall's workaround to fix that:

# Second implementation: work around Pitfall 47.
filter() {
    local prev="" file line
    sed 's/$/:/' | while IFS=: read -r file line; do
        if [[ "$file" != "$prev" ]]; then
            printf '%s:%s\n' "$file" "${line%:}"
        fi
        prev=$file
    done
}

The second implementation adds a colon to the end of every input line (with sed), and then removes it if the line is actually printed (with ${line%:}). This version produces the correct output for every input.

Depending on how the user intends to use this filter, the first implementation might be good enough. This filter is likely to be used only by a human user in an interactive shell environment, where the exact replication of each matching line (including trailing colon) is not as important as just seeing the filenames, and the corner case that triggers the pitfall is likely to be extremely rare.

<- Data structures | Example 0: Filtering a list | Example 1: Modifying a config file ->