How to create a hex dump of file containing only the hex characters without spaces in bash?

LinuxBashHex

Linux Problem Overview


How do I create an unmodified hex dump of a binary file in Linux using bash? The od and hexdump commands both insert spaces in the dump and this is not ideal.

Is there a way to simply write a long string with all the hex characters, minus spaces or newlines in the output?

Linux Solutions


Solution 1 - Linux

xxd -p file

Or if you want it all on a single line:

xxd -p file | tr -d '\n'

Solution 2 - Linux

Format strings can make hexdump behave exactly as you want it to (no whitespace at all, byte by byte):

hexdump -ve '1/1 "%.2x"'

1/1 means "each format is applied once and takes one byte", and "%.2x" is the actual format string, like in printf. In this case: 2-character hexadecimal number, leading zeros if shorter.

Solution 3 - Linux

It seems to depend on the details of the version of od. On OSX, use this:

od -t x1 -An file |tr -d '\n '

(That's print as type hex bytes, with no address. And whitespace deleted afterwards, of course.)

Solution 4 - Linux

Perl one-liner:

perl -e 'local $/; print unpack "H*", <>' file

Solution 5 - Linux

The other answers are preferable, but for a pure Bash solution, I've modified the script in my answer here to be able to output a continuous stream of hex characters representing the contents of a file. (Its normal mode is to emulate hexdump -C.)

Solution 6 - Linux

I think this is the most widely supported version (requiring only POSIX defined tr and od behavior):

cat "$file" | od -v -t x1 -A n | tr -d ' \n'

This uses od to print each byte as hex without address without skipping repeated bytes and tr to delete all spaces and linefeeds in the output. Note that not even the trailing linefeed is emitted here. (The cat is intentional to allow multicore processing where cat can wait for filesystem while od is still processing previously read part. Single core users may want replace that with < "$file" od ... to save starting one additional process.)

Solution 7 - Linux

This code produces a "pure" hex dump string and it runs faster than the all the other examples given. It has been tested on 1GB files filled with binary zeros, and all linefeeds. It is not data content dependent and reads 1MB records instead of lines.

perl -pe 'BEGIN{$/=\1e6} $_=unpack "H*"'

Dozens of timing tests show that for 1GB files, these other methods below are slower. All tests were run writing output to a file which was then verified by checksum. Three 1GB input files were tested: all bytes, all binary zeros, and all LFs.

hexdump -ve '1/1 "%.2x"'                    #  ~10x  slower
od -v -t x1 -An | tr -d "\n "               #  ~15x  slower
xxd -p | tr -d \\n                          #   ~3x  slower
perl -e 'local \$/; print unpack "H*", <>'  # ~1.5x  slower
- this also slurps the whole file into memory

To reverse the process:

perl -pe 'BEGIN{$/=\1e6} $_=pack "H*",$_'

Solution 8 - Linux

tldr;

$ od -t x1 -A n -v <empty.zip | tr -dc '[:xdigit:]' && echo 
504b0506000000000000000000000000000000000000
$

Explanation:

Use the od tool to print single hexadecimal bytes (-t x1) --- without address offsets (-A n) and without eliding repeated "groups" (-v) --- from empty.zip, which has been redirected to standard input. Pipe that to tr which deletes (-d) the complement (-c) of the hexadecimal character set ('[:xdigit:]'). You can optionally print a trailing newline (echo) as I've done here to separate the output from the next shell prompt.

References:

Solution 9 - Linux

You can use Python for this purpose:

python -c "print(open('file.bin','rb').read().hex())"

...where file.bin is your filename.

Explaination:

  1. Open file.bin in rb (read binary) mode.
  2. Read contents (returned as bytes object).
  3. Use bytes method .hex(), which returns hex dump without spaces or new lines.
  4. Print output.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionDavid RaswikView Question on Stackoverflow
Solution 1 - Linuxmark4oView Answer on Stackoverflow
Solution 2 - LinuxMichał TrybusView Answer on Stackoverflow
Solution 3 - LinuxDonal FellowsView Answer on Stackoverflow
Solution 4 - LinuxAlan Haggai AlaviView Answer on Stackoverflow
Solution 5 - LinuxDennis WilliamsonView Answer on Stackoverflow
Solution 6 - LinuxMikko RantalainenView Answer on Stackoverflow
Solution 7 - LinuxGyreView Answer on Stackoverflow
Solution 8 - LinuxrubicksView Answer on Stackoverflow
Solution 9 - LinuxVlad HavriukView Answer on Stackoverflow