Diff for "BashFAQ/019"

Differences between revisions 153 and 156 (spanning 3 versions)

How can I split a file into line ranges, e.g. lines 1-10, 11-20, 21-30?

Some Unix systems provide the split utility for this purpose:

   1 split --lines 10 --numeric-suffixes input.txt output-

For more flexibility you can use sed. The sed command can print e.g. the line number range 1-10:

   1 sed -n -e '1,10p' -e '10q'

This stops sed from printing each line (-n). Instead it only processes the lines in the range 1-10 ("1,10"), and prints them ("p"). The command will quit after reading line 10 ("10q").

We can now use this to print an arbitrary range of a file (specified by line number):

   1 # POSIX shell
   2 file=/etc/passwd
   3 range=10
   4 cur=1
   5 last=$(wc -l < "$file") # count number of lines
   6 chunk=1
   7 while [ $cur -lt $last ]
   8 do
   9     endofchunk=$(($cur + $range - 1))
  10     sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$(printf %04d $chunk)
  11     chunk=$(($chunk + 1))
  12     cur=$(($cur + $range))
  13 done

The previous example uses POSIX arithmetic, which older Bourne shells do not have. In that case the following example should be used instead:

   1 # legacy Bourne shell; assume no printf either
   2 file=/etc/passwd
   3 range=10
   4 cur=1
   5 last=`wc -l < "$file"` # count number of lines
   6 chunk=1
   7 while test $cur -lt $last
   8 do
   9     endofchunk=`expr $cur + $range - 1`
  10     sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$chunk
  11     chunk=`expr $chunk + 1`
  12     cur=`expr $cur + $range`
  13 done

Awk can also be used to produce a more or less equivalent result:

   1 awk -v range=10 '{print > FILENAME "." (int((NR -1)/ range)+1)}' file

CategoryShell

-  ⇤ ← Revision 153 as of 2009-01-19 19:36:45 → 
  Size: 184
  Editor: ip-79-175-202-229
  Comment: -9
+   ← Revision 156 as of 2015-03-05 00:45:47 → ⇥
  Size: 1829
  Editor: izabera
  Comment: syntax hl
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-Wow! Cool design, great info! olives Very cool! http://blcmd.vox.com/library/post/canned-pumpkin-pie-filling-recipes.html frozen felt, http://salinasii.vox.com/tags/ carrots,  Shrug.
+<<Anchor(faq19)>>
== How can I split a file into line ranges, e.g. lines 1-10, 11-20, 21-30? ==
Some Unix systems provide the {{{split}}} utility for this purpose:

{{{#!highlight bash
split --lines 10 --numeric-suffixes input.txt output-
}}}

For more flexibility you can use {{{sed}}}.  The {{{sed}}} command can print e.g. the line number range 1-10:
{{{#!highlight bash
sed -n -e '1,10p' -e '10q'
}}}

This stops {{{sed}}} from printing each line ({{{-n}}}). Instead it only processes the lines in the range 1-10 ("1,10"), and prints them ("p"). The command will quit after reading line 10 ("10q").

We can now use this to print an arbitrary range of a file (specified by line number):

{{{#!highlight bash
# POSIX shell
file=/etc/passwd
range=10
cur=1
last=$(wc -l < "$file") # count number of lines
chunk=1
while [ $cur -lt $last ]
do
    endofchunk=$(($cur + $range - 1))
    sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$(printf %04d $chunk)
    chunk=$(($chunk + 1))
    cur=$(($cur + $range))
done
}}}

The previous example uses POSIX [[ArithmeticExpression|arithmetic]], which older [[BourneShell|Bourne shells]] do not have. In that case the following example should be used instead:

{{{#!highlight bash
# legacy Bourne shell; assume no printf either
file=/etc/passwd
range=10
cur=1
last=`wc -l < "$file"` # count number of lines
chunk=1
while test $cur -lt $last
do
    endofchunk=`expr $cur + $range - 1`
    sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$chunk
    chunk=`expr $chunk + 1`
    cur=`expr $cur + $range`
done
}}}

Awk can also be used to produce a more or less equivalent result:

{{{#!highlight bash
awk -v range=10 '{print > FILENAME "." (int((NR -1)/ range)+1)}' file
}}}

----
CategoryShell