How to pipe input to a Bash while loop and preserve variables after loop ends

BashWhile LoopStdinPipe

Bash Problem Overview


Bash allows to use: cat <(echo "$FILECONTENT")

Bash also allow to use: while read i; do echo $i; done </etc/passwd

to combine previous two this can be used: echo $FILECONTENT | while read i; do echo $i; done

The problem with last one is that it creates sub-shell and after the while loop ends variable i cannot be accessed any more.

My question is:

How to achieve something like this: while read i; do echo $i; done <(echo "$FILECONTENT") or in other words: How can I be sure that i survives while loop?

Please note that I am aware of enclosing while statement into {} but this does not solves the problem (imagine that you want use the while loop in function and return i variable)

Bash Solutions


Solution 1 - Bash

The correct notation for Process Substitution is:

while read i; do echo $i; done < <(echo "$FILECONTENT")

The last value of i assigned in the loop is then available when the loop terminates. An alternative is:

echo $FILECONTENT | 
{
while read i; do echo $i; done
...do other things using $i here...
}

The braces are an I/O grouping operation and do not themselves create a subshell. In this context, they are part of a pipeline and are therefore run as a subshell, but it is because of the |, not the { ... }. You mention this in the question. AFAIK, you can do a return from within these inside a function.


Bash also provides the shopt builtin and one of its many options is:

> lastpipe > > If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.

Thus, using something like this in a script makes the modfied sum available after the loop:

FILECONTENT="12 Name
13 Number
14 Information"
shopt -s lastpipe   # Comment this out to see the alternative behaviour
sum=0
echo "$FILECONTENT" |
while read number name; do ((sum+=$number)); done
echo $sum

Doing this at the command line usually runs foul of 'job control is not active' (that is, at the command line, job control is active). Testing this without using a script failed.

Also, as noted by Gareth Rees in his answer, you can sometimes use a here string:

while read i; do echo $i; done <<< "$FILECONTENT"

This doesn't require shopt; you may be able to save a process using it.

Solution 2 - Bash

Jonathan Leffler explains how to do what you want using process substitution, but another possibility is to use a here string:

while read i; do echo "$i"; done <<<"$FILECONTENT"

This saves a process.

Solution 3 - Bash

This function makes duplicates $NUM times of jpg files (bash)

function makeDups() {
NUM=$1
echo "Making $1 duplicates for $(ls -1 *.jpg|wc -l) files"
ls -1 *.jpg|sort|while read f
do
  COUNT=0
  while [ "$COUNT" -le "$NUM" ]
  do
    cp $f ${f//sm/${COUNT}sm}
    ((COUNT++))
  done
done
}

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionWakan TankaView Question on Stackoverflow
Solution 1 - BashJonathan LefflerView Answer on Stackoverflow
Solution 2 - BashGareth ReesView Answer on Stackoverflow
Solution 3 - BashSteveView Answer on Stackoverflow