Diff for "BashFAQ/100"

Differences between revisions 2 and 21 (spanning 19 versions)

How do I do string manipulations in bash?

Bash can do string operations. LOTS of string operations. This is an introduction to bash string operations for those new to Bash's special tool/feature called Parameter Expansion (PE), with a focus on typical string operations. (Note: much of this material is in FAQ #73. It is being presented here with an approach intended to make it more approachable by people who have not already learned bash PEs.)

Note Bash's PE capability is a lot more powerful than the typical string manipulation calls you may be used to. There are some twists in the road up ahead.

Here is a list of some typical string manipulation functions/subroutines from other languages that you may already be familiar with:

strlen	returns the length of the string
leftstr	returns a string N chars long starting from the left hand side
rightstr	returns a string N chars long starting from the right hand side
midstr	returns a string N chars long starting from an offset K chars from the beginning/end
substr	substitutes all instances of a pattern with a new string
basename	returns the last component of a pathname (everything after the last "/")
dirname	returns everything in the pathname up to, but not including the last "/"

Two that you may not have heard of but would want to use all the time when scripting on *NIX:

getext	returns a filename's extension (eg. "txt", "mp3", "doc", "sxc", "html", etc. ... )
dropext	returns the filename with the extension stripped off the end of the name.

This article will cover how to do all of these using the Bash and will introduce the more powerful actions available with Bash's PEs. Please note there is a BashFAQ about PEs already. FAQ #73 covers a larger scope of PE capabilities, where this one focuses on string operations.

Filename manipulation

Lets say we have a bash variable named fullpath that contains /usr/home/JosephBaldwin/Its_only_Rock_and_Roll.mp3

Often in scripting we want to manipulate certain pieces of the path, like just the file name, which is the last component of the full path. So lets get just the filename from the full path: In *NIX we have the command basename which does this very nicely for us: basename "$fullpath" returns "Its_only_Rock_and_Roll.mp3". In Bash we can do that much faster with this command: echo ${fullpath##*/}

"WHAT? What the heck is that? That's not a command! That's just a bunch of garbage someone made by whacking some of the stranger keys on the keyboard! I mean really! dollar curly pound pound star slash curly What IS that?"

Um, OK - uh, just calm down for a moment. I know it doesn't look like the typical programming language keyword or library call, but consider a language like Perl. See? To shoehorn new features into Bash, you have to find ways to do it without creating keywords (or anything else) that might cause older scripts to break, so all these string manipulation functions got placed inside the syntax from an old sh feature where there just happened to be room for them: "Parameter Expansion".

For a basename we use the PE expression: ${fullpath##*/} which returns "Its_only_Rock_and_Roll.mp3".

To find the dirname we use the PE expression: ${fullpath%/*} which produces "/usr/home/JosephBaldwin".

To drop the filename extension, we use the PE expression: ${fullpath%.*} giving out "/usr/home/JosephBaldwin/Its_only_Rock_and_Roll"

To get the filename's extension we use the PE expression: ${fullpath##*.} generating only "mp3".

To find the strlen, the PE expression: ${#fullpath} finds it, and it's 49.

To get a leftstr, the PE expression: ${fullpath:0:20} grabs the first 20 chars of fullpath to make "usr/home/JosephBaldw".

To perform a rightstr in bash we use the following PE expression: ${fullpath:(-20)} which gets the last 20 chars, "ly_Rock_and_Roll.mp3". The parentheses are needed, although there are a couple other ways to write it that also work.

To perform a midstr in bash we use the following PE expression: ${fullpath:10:20} making "osephBaldwin/Its_onl".

To perform a substr in bash we use the following PE expression: ${fullpath//Rock/Roll} rolling it into "usr/home/JosephBaldwin/Its_only_Roll_and_Roll.mp3".

Why aren't the PE things named more nicely

Can't I just have these a regular functions with nice names?

Not totally generalizable.

What can I do with PE's that I couldn't do with the string functions above?

Todo: offset code examples in code boxes as done in the rest of the wiki.

CategoryShell

-  ⇤ ← Revision 2 as of 2009-06-04 20:21:46 → 
  Size: 2747
  Editor: GeoHump
  Comment:
+   ← Revision 21 as of 2010-04-09 13:05:27 → ⇥
  Size: 4833
  Editor: hide
  Comment: typo fix
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-Bash can do string operations. LOTS of string operations. This is an introduction to bash string operations for those new to Bash's special tool/feature called "Parameter Expansion", (PE),  with a focus on typical string operations.  Note Bash's Parameter Expansion, (PE), capability is a lot more powerful than the typical string manipulation calls you may be used to. There are some twists in the road up ahead.
+<<Anchor(faq100)>>
== How do I do string manipulations in bash? ==
Bash can do string operations. LOTS of string operations. This is an introduction to bash string operations for those new to Bash's special tool/feature called ''Parameter Expansion'' (PE), with a focus on typical string operations. (Note: much of this material is in [[BashFAQ/073|FAQ #73]]. It is being presented here with an approach intended to make it more approachable by people who have not already learned bash PEs.)
-Line 3:
+Line 5:
-Here is a list of some typical string manipulation functions/subroutines that you may already be familiar with:
+Note Bash's PE capability is a lot more powerful than the typical string manipulation calls you may be used to. There are some twists in the road up ahead. :)
-Line 5:
+Line 7:
+Here is a list of some typical string manipulation functions/subroutines from other languages that you may already be familiar with:
-Line 6:
+Line 9:
- * strlen    returns the length of the string
 * leftstr   returns a string N chars long starting from the left hand side
 * rightstr  returns a string N chars long starting from the right hand side
 * midstr    returns a string N chars long starting from offset K chars from the beginning/end
 * substr    returns  copy of the string with all instances matching a patter replaced with a new string
 * basename  returns the last component of a pathname (everything after the last "/")
 * dirname   returns everything in the pathname up to, but not including the last "/"
 * getext    returns a filenames extension
 * dropext   returns the filename without its extension.
+|| '''strlen'''   ||returns the length of the string||
|| '''leftstr'''  ||returns a string N chars long starting from the left hand side||
|| '''rightstr''' ||returns a string N chars long starting from the right hand side||
|| '''midstr'''   ||returns a string N chars long starting from an offset K chars from the beginning/end||
|| '''substr'''   ||substitutes all instances of a pattern with a new string||
|| '''basename''' ||returns the last component of a pathname (everything after the last "/")||
|| '''dirname'''  ||returns everything in the pathname up to, but not including the last "/"||
-Line 16:
+Line 17:
-This article will cover how to do all of these using the Bash PE and will introduce the more powerful actions available with PE's. Pleasenote there is a BasfFaq about PE's already. That FAQ cover's more of the scope of PE capabilities, where this one instead focuses on string operations.
+Two that you may not have heard of but would want to use all the time when scripting on *NIX:
|| '''getext'''  ||returns a filename's extension  (eg. "txt", "mp3", "doc", "sxc", "html",  etc. ... )||
|| '''dropext''' ||returns the filename with the extension stripped off the end of the name.||
-Line 18:
+Line 21:
-lets assume we have a string variable named fullpath whose value is "usr/home/JosephBaldwin/Its_only_Rock_and_Roll.mp3"
+This article will cover how to do all of these using the Bash and will introduce the more powerful actions available with Bash's PEs. Please note there is a BashFAQ about PEs already. [[BashFAQ/073|FAQ #73]] covers a larger scope of PE capabilities, where this one focuses on string operations.
-Line 20:
+Line 23:
+=== Filename manipulation ===
-Line 21:
+Line 25:
+Lets say we have a bash variable named fullpath that contains `/usr/home/JosephBaldwin/Its_only_Rock_and_Roll.mp3`
-Line 22:
+Line 27:
-to perform a basename in bash we use the following PE expression: ${fullpath##*/}
+Often in scripting we want to manipulate certain pieces of the path, like just the file name, which is the last component of the full path.  So lets get just the filename from the full path:  In *NIX we have the command `basename` which does this very nicely for us: 
`basename "$fullpath"` returns "`Its_only_Rock_and_Roll.mp3`".  In Bash we can do that much faster with this command: `echo ${fullpath##*/}`
-Line 24:
+Line 30:
-to perform a dirname in bash we use the following PE expression: ${fullpath%/*} ##Print string with its trailing /component removed
+"WHAT?  What the heck is that? That's not a command!  That's just a bunch of garbage someone made by whacking some of the stranger keys on the keyboard! I mean really! '' dollar curly pound pound star slash curly ''  What IS that?"
-Line 26:
+Line 32:
-to perform a dropext in bash we use the following PE expression: ${fullpath%.*}
+Um, OK - uh, just calm down for a moment.  I know it doesn't look like the typical programming language keyword or library call, but consider a language like Perl.  See? To shoehorn new features into Bash, you have to find ways to do it without creating keywords (or anything else) that might cause older scripts to break, so all these string manipulation functions got placed inside the syntax from an old sh feature where there just happened to be room for them: "Parameter Expansion".
-Line 28:
+Line 34:
-to perform a droppre in bash we use the following PE expression: ${fullpath#*.}
+For a basename we use the PE expression: `${fullpath##*/}` which returns "`Its_only_Rock_and_Roll.mp3`".
-Line 30:
+Line 36:
-to perform a getext  in bash we use the following PE expression: ${fullpath##*.}
+To find the dirname we use the PE expression: `${fullpath%/*}` which produces "`/usr/home/JosephBaldwin`".
-Line 32:
+Line 38:
-to perform a strlen  in bash we use the following PE expression: ${#fullpath}
+To drop the filename extension, we use the PE expression: `${fullpath%.*}` giving out "`/usr/home/JosephBaldwin/Its_only_Rock_and_Roll`"
-Line 34:
+Line 40:
-to perform a leftstr in bash we use the following PE expression: ${fullpath:0:$2}    ## usage leftstr  "somestring"  count
+To get the filename's extension we use the PE expression: `${fullpath##*.}`  generating only "`mp3`".
-Line 36:
+Line 42:
-to perform a rightstr in bash we use the following PE expression: ${fullpath:$(( 0 - $2 ))} ## usage rightstr "somestring"  count
+To find the strlen, the PE expression: `${#fullpath}` finds it, and it's 49.
-Line 38:
+Line 44:
-to perform a midstr  in bash we use the following PE expression: ${fullpath:$2:$3}    usage midstr   "somestring"  StartPos count
+To get a leftstr, the PE expression: `${fullpath:0:20}` grabs the first 20 chars of fullpath to make "`usr/home/JosephBaldw`".
-Line 40:
+Line 46:
-to perform a substr in bash we use the following PE expression: ${fullpath//$2/$3}    substr  "somestring"  "target-pattern" "new-string"
+To perform a rightstr in bash we use the following PE expression: `${fullpath:(-20)}` which gets the last 20 chars, "`ly_Rock_and_Roll.mp3`".  The parentheses are needed, although there are a couple other ways to write it that also work.
-Line 42:
+Line 48:
+To perform a midstr in bash we use the following PE expression: `${fullpath:10:20}` making "`osephBaldwin/Its_onl`".
-Line 43:
+Line 50:
+To perform a substr in bash we use the following PE expression: `${fullpath//Rock/Roll}` rolling it into "`usr/home/JosephBaldwin/Its_only_Roll_and_Roll.mp3`".
-Line 44:
+Line 52:
+=== Why aren't the PE things named more nicely ===

=== Can't I just have these a regular functions with nice names? ===

Not totally generalizable.

=== What can I do with PE's that I couldn't do with the string functions above? ===

Todo: offset code examples in code boxes as done in the rest of the wiki.