GZip every file separately

LinuxBashGzip

Linux Problem Overview


How can we GZip every file separately?

I don't want to have all of the files in a big tar.

Linux Solutions


Solution 1 - Linux

You can use gzip *


Note:

  • This will zip each file individually and DELETE the original.
  • Use -k (--keep) option to keep the original files.
  • This may not work if you have a huge number of files due to limits of the shell
  • To run gzip in parallel see @MarkSetchell's answer below.

Solution 2 - Linux

Easy and very fast answer that will use all your CPU cores in parallel:

parallel gzip ::: *

GNU Parallel is a fantastic tool that should be used far more in this world where CPUs are only getting more cores rather than more speed. There are loads of examples that we would all do well to take 10 minutes to read... here

Solution 3 - Linux

After seven years, this highly upvoted comment still doesn't have its own full-fledged answer, so I'm promoting it now:

gzip -r .

This has two advantages over the currently accepted answer: it works recursively if there are any subdirectories, and it won't fail from Argument list too long if the number of files is very large.

Solution 4 - Linux

If you want to gzip every file recursively, you could use find piped to xargs:

$ find . -type f -print0 | xargs -0r gzip

Solution 5 - Linux

Try a loop

$ for file in *; do gzip "$file"; done

Solution 6 - Linux

Or, if you have pigz (gzip utility that parallelizes compression over multiple processors and cores)

pigz *

Solution 7 - Linux

The following command can run multiple times inside a directory (without "already has .gz suffix" warnings) to gzip whatever is not already gzipped.

find . -maxdepth 1 -type f ! -name '*.gz' -exec gzip "{}" \;

A more useful example of utilizing find is when you want to gzip rolling logs. E.g. you want every day or every month to gzip rolled logs but not current logs.

# Considering that current logs end in .log and 
# rolled logs end in .log.[yyyy-mm-dd] or .log.[number]
find . -maxdepth 1 -type f ! -name '*.gz' ! -name '*.log' -exec gzip "{}" \;

Solution 8 - Linux

fyi, this will help to overwrite if an existing gz file along with creating few gz file if it is not present:

find . -type f | grep "in case any specific" | grep -E -v "*.gz$" | xargs -n1 -P8 sh -c 'yes | gzip --force --best -f $0'

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionTonioView Question on Stackoverflow
Solution 1 - LinuxCourtney FaulknerView Answer on Stackoverflow
Solution 2 - LinuxMark SetchellView Answer on Stackoverflow
Solution 3 - LinuxleekaiintheskyView Answer on Stackoverflow
Solution 4 - LinuxBuddyView Answer on Stackoverflow
Solution 5 - LinuxFederico GiorgiView Answer on Stackoverflow
Solution 6 - LinuxitsmisterbrownView Answer on Stackoverflow
Solution 7 - LinuxMarinos AnView Answer on Stackoverflow
Solution 8 - LinuxVadiraj k.sView Answer on Stackoverflow