How can I replace multiple empty lines with a single empty line in bash?
RegexBashRegex Problem Overview
I have a file that contains:
something
something else
something else again
I need a bash command, sed/grep w.e that will produce the following output
something
something else
something else again
In other words, I need to replace multiple blank lines with just a single blank line. grep/sed are line based. I've never found a BASH solution that would work on multi-line regex patterns.
Regex Solutions
Solution 1 - Regex
For BSD-derived systems (including GNU):
You just need cat with the -s option which causes it to remove repeated empty lines from its output:
cat -s
From man page: -s --squeeze-blank: suppress repeated empty output lines.
Solution 2 - Regex
I just solved this problem by sed. Even if this is a 7 year old question, someone may find this helpful, so I am writing my solution by sed here:
sed 'N;/^\n$/D;P;D;'
Solution 3 - Regex
grep -A1 . <yourfile> | grep -v "^--$"
This grep solution works assuming you want the following:
Input
line1line2 line3
line4
line5
Output
line1line2 line3
line4
line5
Solution 4 - Regex
Actually, if you replace multiple newlines with a single newline, the output would be:
something
something else
something else again
You can achieve this by:
sed /^$/d FILE
Solution 5 - Regex
A solution with awk, which replaces several blank lines with a single blank line:
awk 'BEGIN{bl=0}/^$/{bl++;if(bl==1)print;else next}/^..*$/{bl=0;print}' myfile
Solution 6 - Regex
Usually, if I find that sed can't do something I need, I turn to awk:
awk '
BEGIN {
blank = 0;
}
/^[[:blank:]]*$/ {
if (!blank) {
print;
}
blank = 1;
next;
}
{
print;
blank = 0;
}' file
Solution 7 - Regex
If someone want use perl
perl -00pe0 < file
will do the same, as cat -s :)
Solution 8 - Regex
Use awk:
awk '{ /^\s*$/?b++:b=0; if (b<=1) print }' file
Breakdown:
/^\s*$/?b++:b=0
- ? : the ternary operator
- /^\s*$/ matches a blank line
- b variable that counts consecutive blank lines (b++).
however, if the current line is non-blank, b is reset to 0.
if (b<=1) print
print if the current line is non-blank (b==0)
or if there is only one blank line (b==1).
By adjusting the regex, you can generalize it to other scenarios like squeezing multiple blank lines (">") in email: https://stackoverflow.com/a/59189823/12483961
Solution 9 - Regex
This uses marco's solution on multiple files:
for i in *; do FILE=$(cat -s "$i"); echo "$FILE" > "$i"; done
Solution 10 - Regex
Use python:
s = file("filename.txt").read()
while "\n\n\n" in s: s = s.replace("\n\n\n", "\n\n")
import sys
sys.stdout.write(s)
Solution 11 - Regex
Python, with regular expression:
import re
import sys
sys.stdout.write(re.sub('\n{2,}','\n\n', sys.stdin.read()))
Solution 12 - Regex
Super easy to do with vim. Just open the file and type the following:
:%s/\n\n\n*/\r\r/
That will reduce all blocks of more than 2 new lines to 2 new lines. Hope this helps!
Solution 13 - Regex
I take it that you'll probably want to remove lines that only have whitespace.
That can be done with:
sed /^[:space:]*$/d FILE
Solution 14 - Regex
Pipelining it to |uniq may be solution (if other than empty lines don't duplicate)