Differences between revisions 6 and 7
Revision 6 as of 2008-11-22 21:54:31
Size: 1968
Editor: GreyCat
Comment: first-line and }}}
Revision 7 as of 2009-10-05 00:01:36
Size: 2041
Editor: c-67-174-7-187
Comment: added the :w option to the vim line to indicate a save. Also added another way via col < infile > outfile command to remove M$. Sorry about my formatting.
Deletions are marked like this. Additions are marked like this.
Line 46: Line 46:
 * In {{{vim}}}, you can use {{{:set fileformat=unix}}} to do it.  * You can also use col <input.txt > output.txt
* In {{{vim}}}, you can use {{{:set fileformat=unix}}} to do it and save it with a ":w".

How do I convert a file from DOS format to UNIX format (remove CRs from CR-LF line terminators)?

Carriage return characters (CRs) are used in line ending markers on some systems. There are three different kinds of line endings in common use:

  • Unix systems use Line Feeds (LFs) only.
  • MS-DOS and Windows systems use CR-LF pairs.
  • Old Macintosh systems use CRs only.

If you're running a script on a Unix system, the line endings need to be Unix ones (LFs only), or you will have problems. You can check the kind of line endings in use by running:

cat -e yourscript

If you see something like this:

another command^M$

then you need to remove the CRs. There are a plethora of ways to do this.

All these are from the sed one-liners page:

sed 's/.$//' dosfile              # assumes that all lines end with CR/LF
sed 's/^M$//' dosfile             # in bash/tcsh, press Ctrl-V then Ctrl-M
sed 's/\x0D$//' dosfile           # GNUism - does not work with Unix sed!

If you want to remove all CRs regardless of whether they are at the end of a line, you can use tr:

tr -d '\r' < dosfile

If you want to use the second sed example above, but without embedding a literal CR into your script:

sed $'s/\r$//' dosfile            # BASH only

All of the previous examples write the modified file to standard output. Redirect the output to a new file, and then mv it over top of the original.

There are many more ways:

  • Some systems have a dos2unix command which can do this. Or recode, or fromdos.

  • You can also use col <input.txt > output.txt

  • In vim, you can use :set fileformat=unix to do it and save it with a ":w".

  • You can use Perl:
    •   perl -pi -e 's/\r\n/\n/' filename
    This has the advantage of overwriting the original file, so you don't have to mess with temporary files.

BashFAQ/052 (last edited 2022-01-30 01:59:53 by larryv)