Can $() always replace backticks for command substitution?

BashSyntax

Bash Problem Overview


The following code fails to generate binary numbers if backticks are replaced by dollar-parenthesis syntax:

#!/bin/bash
rm test.bin 2>/dev/null
for character in {0..255}
do
    char=`printf '\\\\x'"%02x" $character`
    printf "$char" >> test.bin
done
hexdump -C test.bin

Result:

00000000  00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
00000010  10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
00000020  20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f  | !"#$%&'()*+,-./|
00000030  30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  |0123456789:;<=>?|
00000040  40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  |@ABCDEFGHIJKLMNO|
00000050  50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  |PQRSTUVWXYZ[\]^_|
00000060  60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  |`abcdefghijklmno|
00000070  70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  |pqrstuvwxyz{|}~.|
00000080  80 81 82 83 84 85 86 87  88 89 8a 8b 8c 8d 8e 8f  |................|
00000090  90 91 92 93 94 95 96 97  98 99 9a 9b 9c 9d 9e 9f  |................|
000000a0  a0 a1 a2 a3 a4 a5 a6 a7  a8 a9 aa ab ac ad ae af  |................|
000000b0  b0 b1 b2 b3 b4 b5 b6 b7  b8 b9 ba bb bc bd be bf  |................|
000000c0  c0 c1 c2 c3 c4 c5 c6 c7  c8 c9 ca cb cc cd ce cf  |................|
000000d0  d0 d1 d2 d3 d4 d5 d6 d7  d8 d9 da db dc dd de df  |................|
000000e0  e0 e1 e2 e3 e4 e5 e6 e7  e8 e9 ea eb ec ed ee ef  |................|
000000f0  f0 f1 f2 f3 f4 f5 f6 f7  f8 f9 fa fb fc fd fe ff  |................|

That's ok so far. Let's replace backticks and see what we get:

#!/bin/bash
rm test.bin 2>/dev/null
for character in {0..255}
do
    char=$(printf '\\\\x'"%02x" $character)
    printf "$char" >> test.bin
done
hexdump -C test.bin

Result:

00000000  5c 78 30 30 5c 78 30 31  5c 78 30 32 5c 78 30 33  |\x00\x01\x02\x03|
00000010  5c 78 30 34 5c 78 30 35  5c 78 30 36 5c 78 30 37  |\x04\x05\x06\x07|
00000020  5c 78 30 38 5c 78 30 39  5c 78 30 61 5c 78 30 62  |\x08\x09\x0a\x0b|
00000030  5c 78 30 63 5c 78 30 64  5c 78 30 65 5c 78 30 66  |\x0c\x0d\x0e\x0f|
00000040  5c 78 31 30 5c 78 31 31  5c 78 31 32 5c 78 31 33  |\x10\x11\x12\x13|
00000050  5c 78 31 34 5c 78 31 35  5c 78 31 36 5c 78 31 37  |\x14\x15\x16\x17|
00000060  5c 78 31 38 5c 78 31 39  5c 78 31 61 5c 78 31 62  |\x18\x19\x1a\x1b|
00000070  5c 78 31 63 5c 78 31 64  5c 78 31 65 5c 78 31 66  |\x1c\x1d\x1e\x1f|
.
.

While I prefer dollar-parenthesis syntax it appears to fail in this case but why ? Credits for the code snippet: http://code.activestate.com/recipes/578441-a-building-block-bash-binary-file-manipulation

Bash Solutions


Solution 1 - Bash

You are running into one of the reasons that $() is preferred to the backtick notation. The shell parsing of $() is more consistent (as it introduces a new parsing context as I understand it).

So your escaping, while correct for the backtick code, is excessive for the $() code.

Try this:

$ : > test.bin; for character in {0..255}
do
    char=$(printf '\\x'"%02x" $character)
    printf "$char" >> test.bin
done; hexdump -C test.bin
00000000  00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
00000010  10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
00000020  20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f  | !"#$%&'()*+,-./|
00000030  30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  |0123456789:;<=>?|
00000040  40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  |@ABCDEFGHIJKLMNO|
00000050  50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  |PQRSTUVWXYZ[\]^_|
00000060  60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  |`abcdefghijklmno|
00000070  70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  |pqrstuvwxyz{|}~.|
00000080  80 81 82 83 84 85 86 87  88 89 8a 8b 8c 8d 8e 8f  |................|
00000090  90 91 92 93 94 95 96 97  98 99 9a 9b 9c 9d 9e 9f  |................|
000000a0  a0 a1 a2 a3 a4 a5 a6 a7  a8 a9 aa ab ac ad ae af  |................|
000000b0  b0 b1 b2 b3 b4 b5 b6 b7  b8 b9 ba bb bc bd be bf  |................|
000000c0  c0 c1 c2 c3 c4 c5 c6 c7  c8 c9 ca cb cc cd ce cf  |................|
000000d0  d0 d1 d2 d3 d4 d5 d6 d7  d8 d9 da db dc dd de df  |................|
000000e0  e0 e1 e2 e3 e4 e5 e6 e7  e8 e9 ea eb ec ed ee ef  |................|
000000f0  f0 f1 f2 f3 f4 f5 f6 f7  f8 f9 fa fb fc fd fe ff  |................|
00000100

A little more clearly compare this

$ printf %s\\n `printf %s "\\\\ff"`
\ff
$ printf %s\\n `printf %s '\\\\ff'`
\\ff

to this

$ printf %s\\n $(printf %s "\\\\ff")
\\ff
$ printf %s\\n $(printf %s '\\\\ff')
\\\\ff

Solution 2 - Bash

This is the difference:

echo `echo '\\'`
\

echo $(echo '\\')
\\

From the manual, Command substitution section:

> When the old-style backquoted form of substitution is used, backslash retains its literal meaning except when followed by "$", "`", or "".

> When using the "$(COMMAND)" form, all characters between the parentheses make up the command; none are treated specially.

$() doesn't need extra escaping for \. Use:

char=$(printf '\\x'"%02x" $character)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionajaaskelView Question on Stackoverflow
Solution 1 - BashEtan ReisnerView Answer on Stackoverflow
Solution 2 - BashKaroly HorvathView Answer on Stackoverflow