How to apply shell command to each line of a command output?

Bash

Bash Problem Overview


Suppose I have some output from a command (such as ls -1):

a
b
c
d
e
...

I want to apply a command (say echo) to each one, in turn. E.g.

echo a
echo b
echo c
echo d
echo e
...

What's the easiest way to do that in bash?

Bash Solutions


Solution 1 - Bash

It's probably easiest to use xargs. In your case:

ls -1 | xargs -L1 echo

The -L flag ensures the input is read properly. From the man page of xargs:

-L number
    Call utility for every number non-empty lines read. 
    A line ending with a space continues to the next non-empty line. [...]

Solution 2 - Bash

You can use a basic prepend operation on each line:

ls -1 | while read line ; do echo $line ; done

Or you can pipe the output to sed for more complex operations:

ls -1 | sed 's/^\(.*\)$/echo \1/'

Solution 3 - Bash

You can use a http://www.cyberciti.biz/faq/bash-for-loop/">for loop:

for file in * ; do
echo "$file"
done

Note that if the command in question accepts multiple arguments, then using http://www.opengroup.org/onlinepubs/009695399/utilities/xargs.html">xargs</a> is almost always more efficient as it only has to spawn the utility in question once instead of multiple times.

Solution 4 - Bash

You actually can use sed to do it, provided it is GNU sed.

... | sed 's/match/command \0/e'

How it works:

  1. Substitute match with command match
  2. On substitution execute command
  3. Replace substituted line with command output.

Solution 5 - Bash

for s in `cmd`; do echo $s; done

If cmd has a large output:

cmd | xargs -L1 echo

Solution 6 - Bash

xargs fails with with backslashes, quotes. It needs to be something like

ls -1 |tr \\n \\0 |xargs -0 -iTHIS echo "THIS is a file."

xargs -0 option:

> -0, --null Input items are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when input items might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode.

ls -1 terminates the items with newline characters, so tr translates them into null characters.

This approach is about 50 times slower than iterating manually with for ... (see Michael Aaron Safyans answer) (3.55s vs. 0.066s). But for other input commands like locate, find, reading from a file (tr \\n \\0 <file) or similar, you have to work with xargs like this.

Solution 7 - Bash

i like to use gawk for running multiple commands on a list, for instance

ls -l | gawk '{system("/path/to/cmd.sh "$1)}'

however the escaping of the escapable characters can get a little hairy.

Solution 8 - Bash

Better result for me:

ls -1 | xargs -L1 -d "\n" CMD

Solution 9 - Bash

A solution that works with filenames that have spaces in them, is:

ls -1 | xargs -I %s echo %s

The following is equivalent, but has a clearer divide between the precursor and what you actually want to do:

ls -1 | xargs -I %s -- echo %s

Where echo is whatever it is you want to run, and the subsequent %s is the filename.

Thanks to Chris Jester-Young's answer on a duplicate question.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAlex BudovskiView Question on Stackoverflow
Solution 1 - BashMichael MrozekView Answer on Stackoverflow
Solution 2 - BashTrey HunnerView Answer on Stackoverflow
Solution 3 - BashMichael Aaron SafyanView Answer on Stackoverflow
Solution 4 - BashŁukasz DanilukView Answer on Stackoverflow
Solution 5 - BashMarcelo CantosView Answer on Stackoverflow
Solution 6 - Bashphil294View Answer on Stackoverflow
Solution 7 - BashChrisView Answer on Stackoverflow
Solution 8 - BashAndrej PandovichView Answer on Stackoverflow
Solution 9 - BashbaluptonView Answer on Stackoverflow