Differences between revisions 1 and 13 (spanning 12 versions)
Revision 1 as of 2007-05-02 23:44:22
Size: 469
Editor: redondos
Comment:
Revision 13 as of 2012-10-16 20:59:44
Size: 2135
Editor: e36freak
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
[[Anchor(faq52)]]
== How do I convert a file in DOS format to UNIX format. ( Remove CRLF line terminators ) ==
<<Anchor(faq52)>>
== How do I convert a file from DOS format to UNIX format (remove CRs from CR-LF line terminators)? ==
Carriage return characters (CRs) are used in line ending markers on some systems. There are three different kinds of line endings in common use:
Line 4: Line 5:
All these are from the sed one-liners page  *Unix systems use Line Feeds (LFs) only.
 *MS-DOS and Windows systems use CR-LF pairs.
 *Old Macintosh systems use CRs only.

If you're running a script on a Unix system, the line endings need to be Unix ones (LFs only), or you will have problems. You can check the kind of line endings in use by running:
Line 6: Line 11:
sed 's/.$//' dosfile # assumes that all lines end with CR/LF
sed 's/^M$//' dosfile # in bash/tcsh, press Ctrl-V then Ctrl-M
sed 's/\x0D$//' dosfile
cat -e yourscript
Line 11: Line 14:
Some distributions have ''dos2unix'' command which can do this. In vim, you can use '':set fileformat=unix'' If you see something like this:
{{{
command^M$
^M$
another command^M$
}}}

then you need to remove the CRs. There are a plethora of ways to do this.

To remove them from a file, `ex` is a good standard way to do it:

{{{
ex -sc $'%s/\r$//e|x' file
}}}

There are many more ways:
 * Some systems have a {{{dos2unix}}} command which can do this. Or {{{recode}}}, or {{{fromdos}}}.
 * You can also use col <input.txt > output.txt
 * In {{{vim}}}, you can use {{{:set fileformat=unix}}} to do it and save it with a ":w".
 * You can use Perl:
  {{{
  perl -pi -e 's/\r\n/\n/' filename
  }}}
 This has the advantage of overwriting the original file, so you don't have to mess with temporary files.

----

Another way to check it:
{{{
file yourscript
}}}
The output tells you whether the ASCII text has some CR, if that's the case. Note: this is only true on GNU/Linux. On other operating systems, the result of `file` is unpredictable, except that it should contain the word "text" somewhere in the output if the result "kind of looks like a text file of some sort, maybe".
{{{
imadev:~$ printf 'DOS\r\nline endings\r\n' > foo
imadev:~$ file foo
foo: commands text
arc3:~$ file foo
foo: ASCII text, with CRLF line terminators
}}}

And another way to fix it:
{{{
nano -w yourscript
}}}
Type Ctrl-O and before confirming, type Alt-D (DOS) or Alt-M (Mac) to change the format.

Or, if available, you can use the program `dos2unix`, sometimes also called `tofrodos`.

How do I convert a file from DOS format to UNIX format (remove CRs from CR-LF line terminators)?

Carriage return characters (CRs) are used in line ending markers on some systems. There are three different kinds of line endings in common use:

  • Unix systems use Line Feeds (LFs) only.
  • MS-DOS and Windows systems use CR-LF pairs.
  • Old Macintosh systems use CRs only.

If you're running a script on a Unix system, the line endings need to be Unix ones (LFs only), or you will have problems. You can check the kind of line endings in use by running:

cat -e yourscript

If you see something like this:

command^M$
^M$
another command^M$

then you need to remove the CRs. There are a plethora of ways to do this.

To remove them from a file, ex is a good standard way to do it:

ex -sc $'%s/\r$//e|x' file

There are many more ways:

  • Some systems have a dos2unix command which can do this. Or recode, or fromdos.

  • You can also use col <input.txt > output.txt

  • In vim, you can use :set fileformat=unix to do it and save it with a ":w".

  • You can use Perl:
    •   perl -pi -e 's/\r\n/\n/' filename
    This has the advantage of overwriting the original file, so you don't have to mess with temporary files.


Another way to check it:

file yourscript

The output tells you whether the ASCII text has some CR, if that's the case. Note: this is only true on GNU/Linux. On other operating systems, the result of file is unpredictable, except that it should contain the word "text" somewhere in the output if the result "kind of looks like a text file of some sort, maybe".

imadev:~$ printf 'DOS\r\nline endings\r\n' > foo
imadev:~$ file foo
foo:            commands text
arc3:~$ file foo
foo: ASCII text, with CRLF line terminators

And another way to fix it:

nano -w yourscript

Type Ctrl-O and before confirming, type Alt-D (DOS) or Alt-M (Mac) to change the format.

Or, if available, you can use the program dos2unix, sometimes also called tofrodos.

BashFAQ/052 (last edited 2022-01-30 01:59:53 by larryv)