How to assign a heredoc value to a variable in Bash?
BashHeredocBash Problem Overview
I have this multi-line string (quotes included):
abc'asdf"
$(dont-execute-this)
foo"bar"''
How would I assign it to a variable using a heredoc in Bash?
I need to preserve newlines.
I don't want to escape the characters in the string, that would be annoying...
Bash Solutions
Solution 1 - Bash
You can avoid a useless use of cat
and handle mismatched quotes better with this:
$ read -r -d '' VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
If you don't quote the variable when you echo it, newlines are lost. Quoting it preserves them:
$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''
If you want to use indentation for readability in the source code, use a dash after the less-thans. The indentation must be done using only tabs (no spaces).
$ read -r -d '' VAR <<-'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''
If, instead, you want to preserve the tabs in the contents of the resulting variable, you need to remove tab from IFS
. The terminal marker for the here doc (EOF
) must not be indented.
$ IFS='' read -r -d '' VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''
Tabs can be inserted at the command line by pressing Ctrl-V Tab. If you are using an editor, depending on which one, that may also work or you may have to turn off the feature that automatically converts tabs to spaces.
Solution 2 - Bash
Use $() to assign the output of cat
to your variable like this:
VAR=$(cat <<'END_HEREDOC'
abc'asdf"
$(dont-execute-this)
foo"bar"''
END_HEREDOC
)
# this will echo variable with new lines intact
echo "$VAR"
# this will echo variable without new lines (changed to space character)
echo $VAR
Making sure to delimit starting END_HEREDOC with single-quotes.
Note that ending heredoc delimiter END_HEREDOC
must be alone on the line (hence ending parenthesis is on the next line).
Thanks to @ephemient
for the answer.
Solution 3 - Bash
this is variation of Dennis method, looks more elegant in the scripts.
function definition:
define(){ IFS='\n' read -r -d '' ${1} || true; }
usage:
define VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
echo "$VAR"
enjoy
p.s. made a 'read loop' version for shells that do not support read -d
. should work with set -eu
and unpaired backticks, but not tested very well:
define(){ o=; while IFS="\n" read -r a; do o="$o$a"'
'; done; eval "$1=\$o"; }
Solution 4 - Bash
VAR=<<END
abc
END
doesn't work because you are redirecting stdin to something that doesn't care about it, namely the assignment
export A=`cat <<END
sdfsdf
sdfsdf
sdfsfds
END
` ; echo $A
works, but there's a back-tic in there that may stop you from using this. Also, you should really avoid using backticks, it's better to use the command substitution notation $(..)
.
export A=$(cat <<END
sdfsdf
sdfsdf
sdfsfds
END
) ; echo $A
Solution 5 - Bash
> There is still no solution that preserves newlines.
This is not true - you're probably just being misled by the behaviour of echo:
echo $VAR # strips newlines
echo "$VAR" # preserves newlines
Solution 6 - Bash
Branching off Neil's answer, you often don't need a var at all, you can use a function in much the same way as a variable and it's much easier to read than the inline or read
-based solutions.
$ complex_message() {
cat <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
}
$ echo "This is a $(complex_message)"
This is a abc'asdf"
$(dont-execute-this)
foo"bar"''
Solution 7 - Bash
An array is a variable, so in that case mapfile will work
mapfile y <<'z'
abc'asdf"
$(dont-execute-this)
foo"bar"''
z
Then you can print like this
printf %s "${y[@]}"
Solution 8 - Bash
assign a heredoc value to a variable
VAR="$(cat <<'VAREOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
VAREOF
)"
used as an argument of a command
echo "$(cat <<'SQLEOF'
xxx''xxx'xxx'xx 123123 123123
abc'asdf"
$(dont-execute-this)
foo"bar"''
SQLEOF
)"
Solution 9 - Bash
Thanks to dimo414's answer, this shows how his great solution works, and shows that you can have quotes and variables in the text easily as well:
example output
$ ./test.sh
The text from the example function is:
Welcome dev: Would you "like" to know how many 'files' there are in /tmp?
There are " 38" files in /tmp, according to the "wc" command
test.sh
#!/bin/bash
function text1()
{
COUNT=$(\ls /tmp | wc -l)
cat <<EOF
$1 Would you "like" to know how many 'files' there are in /tmp?
There are "$COUNT" files in /tmp, according to the "wc" command
EOF
}
function main()
{
OUT=$(text1 "Welcome dev:")
echo "The text from the example function is: $OUT"
}
main
Solution 10 - Bash
I found myself having to read a string with NULL in it, so here is a solution that will read anything you throw at it. Although if you actually are dealing with NULL, you will need to deal with that at the hex level.
$ cat > read.dd.sh
read.dd() {
buf=
while read; do
buf+=$REPLY
done < <( dd bs=1 2>/dev/null | xxd -p )
printf -v REPLY '%b' $( sed 's/../ \\\x&/g' <<< $buf )
}
Proof:
$ . read.dd.sh
$ read.dd < read.dd.sh
$ echo -n "$REPLY" > read.dd.sh.copy
$ diff read.dd.sh read.dd.sh.copy || echo "File are different"
$
HEREDOC example (with ^J, ^M, ^I):
$ read.dd <<'HEREDOC'
> (TAB)
> (SPACES)
(^J)^M(^M)
> DONE
>
> HEREDOC
$ declare -p REPLY
declare -- REPLY=" (TAB)
(SPACES)
(^M)
DONE
"
$ declare -p REPLY | xxd
0000000: 6465 636c 6172 6520 2d2d 2052 4550 4c59 declare -- REPLY
0000010: 3d22 0928 5441 4229 0a20 2020 2020 2028 =".(TAB). (
0000020: 5350 4143 4553 290a 285e 4a29 0d28 5e4d SPACES).(^J).(^M
0000030: 290a 444f 4e45 0a0a 220a ).DONE
Solution 11 - Bash
Here's a way to do it that is (imho) quite elegant and avoids a UUOC:
VAR=$(sed -e 's/[ ]*\| //g' -e '1d;$d' <<'--------------------'
|
| <!DOCTYPE html>
| <html>
| <head>
| <script src='script.js'></script>
| </head>
| <body>
| <span id='hello-world'></span>
| </body>
| </html>
|
--------------------
)
The '|' characters define the margin, and only the whitespace to the right of the margin is respected in the printed string. The '1d;$d'
strips the first and last line, which are just added as a top and bottom margin around the content. Everything can be indented to whatever level you like, except the HEREDOC delimiter, which in this case is just a bunch of hyphens.
echo "$VAR"
# prints
<!DOCTYPE html>
<html>
<head>
<script src='script.js'></script>
</head>
<body>
<span id='hello-world'></span>
</body>
</html>
Solution 12 - Bash
$TEST="ok"
read MYTEXT <<EOT
this bash trick
should preserve
newlines $TEST
long live perl
EOT
echo -e $MYTEXT