Reading null delimited strings through a Bash loop
BashDelimiterNull CharacterBash Problem Overview
I want to iterate through a list of files without caring about what characters the filenames might contain, so I use a list delimited by null characters. The code will explain things better.
# Set IFS to the null character to hopefully change the for..in
# delimiter from the space character (sadly does not appear to work).
IFS=$'\0'
# Get null delimited list of files
filelist="`find /some/path -type f -print0`"
# Iterate through list of files
for file in $filelist ; do
# Arbitrary operations on $file here
done
The following code works when reading from a file, but I need to read from a variable containing text.
while read -d $'\0' line ; do
# Code here
done < /path/to/inputfile
Bash Solutions
Solution 1 - Bash
The preferred way to do this is using process substitution
while IFS= read -r -d $'\0' file; do
# Arbitrary operations on "$file" here
done < <(find /some/path -type f -print0)
If you were hell-bent on parsing a bash variable in a similar manner, you can do so as long as the list is not NUL-terminated.
Here is an example of bash var holding a tab-delimited string
$ var=$(echo -ne "foo\tbar\tbaz\t");
$ while IFS= read -r -d $'\t' line ; do \
echo "#$line#"; \
done <<<"$var"
#foo#
#bar#
#baz#
Solution 2 - Bash
Use env -0
to output the assignments by the zero byte.
env -0 | while IFS='' read -d '' line ; do
var=${line%%=*}
value=${line#*=}
echo "Variable '$var' has the value '$value'"
done
Solution 3 - Bash
Pipe them to xargs -0
:
files="$( find ./ -iname 'file*' -print0 | xargs -0 )"
> -0, --null > Input items are terminated by a null character instead of > by whitespace, and the quotes and backslash are not > special (every character is taken literally).
Solution 4 - Bash
In terms of readability and maintainability a bash function might be cleaner:
An example that converts MOV
files to MP4
using ffmpeg
(works with files containing spaces and special characters):
#!/usr/bin/env bash
do_convert () {
new_file="${1/.mov/.mp4}"
ffmpeg -i "$1" "$new_file" && rm "$1"
}
export -f do_convert # needed to make the function visible inside xargs
find . -iname '*.mov' -print0 | xargs -0 -I {} bash -c 'do_convert "{}"' _ {}
Does not apply to the OP's question but in case your input is generated by find
then there is no need to pipe via xargs -0
as find
is perfectly capable of handling non-ascii characters and spaces in file names. If you don't care about readability and maintainability then the command above can be simplified to:
find . -type f -iname "*.mov" -exec bash -c 'ffmpeg -i "${1}" "${1%.*}.mp4" && rm "${1}"' _ {} \;
Solution 5 - Bash
I tried working with the bash examples above, and finally gave up, and used Python, which worked the first time. For me it turned out the problem was simpler outside the shell. I know this is possibly off topic of a bash solution, but I'm posting it here anyway in case others want an alternative.
import sh
import path
files = path.Path(".").files()
for x in files:
sh.cp("--reflink=always", x, "UUU00::%s"%(x.basename(),))
sh.cp("--reflink=always", x, "UUU01::%s"%(x.basename(),))