Insert newline (\n) using sed

BashSedFreebsd

Bash Problem Overview


I am trying to scrub some lists into a properly formatted CSV file for database import.

My starting file, looks something like this with what is supposed to be each "line" spanning multiple lines like below

Mr. John Doe
Exclusively Stuff, 186 
Caravelle Drive, Ponte Vedra
33487. 

I created a sed script that cleans up the file (there's lots of "dirty" formatting like double spaces and spaces before/after commas). The problem is the Zip with the period. I would like to change that period for a new line, but I cannot get it to work.

The command that I use is:

sed -E -f scrub.sed test.txt

and the scrub.sed script is as follows:

:a
N
s|[[:space:]][[:space:]]| |g
s|,[[:space:]]|,|g
s|[[:space:]],|,|g
s|\n| |g
s|[[:space:]]([0-9]{5})\.|,FL,\1\n |g
$!ba

What I get is

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487n 

If figured that the Zip+.(period) would be a great "delimiter" to use the substitution on and while I can find it, I can't seem to tell it to put a newline there.

Most of the things I found online are about replacing the newline with something else (usually deleting them), but not much on replacing with a newline. I did find this, but it didn't work: https://stackoverflow.com/questions/9473060/how-to-insert-newline-character-after-comma-in-with-sed

Is there something I am missing?

Update:

I edited my scrub.sed file putting the literal new line as instucted. It still doesn't work

:a
N
s|[[:space:]][[:space:]]| |g
s|,[[:space:]]|,|g
s|[[:space:]],|,|g
s|\n| |g
s|[[:space:]]([0-9]{5})\.|,FL,\1\
|g
$!ba

What I get is (everything on one line):

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487 Mrs. Jane Smith,Props and Stuff,123 Main Drive,Jacksonville,FL,336907  

My expected output should be:

Mr. John Doe,Exclusively Stuff,186 Caravelle Drive,Ponte Vedra,FL,33487
Mrs. Jane Smith,Props and Stuff,123 Main Drive,Jacksonville,FL,336907  

Bash Solutions


Solution 1 - Bash

The sed on BSD does not support the \n representation of a new line (turning it into a literal n):

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1\n next line/'
123n next line

GNU sed does support the \n representation:

$ echo "123." | gsed -E 's/([[:digit:]]*)\./\1\nnext line/'
123
next line

Alternatives are:

Use a single character delimiter that you then use tr translate into a new line:

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1|next line/' | tr '|' '\n'
123
next line

Or use an escaped literal new line in your sed script:

$ echo "123." | sed -E 's/([[:digit:]]*)\./\1\
next line/'
123
next line

Or define a new line:

POSIX:

nl='
'

BASH / zsh / others that support ANSI C quoting:

nl=$'\n'

And then use sed with appropriate quoting and escapes to insert the literal \n:

echo "123." | sed 's/\./'"\\${nl}"'next line/'
123
next line

Or use awk:

$ echo "123." | awk '/^[[:digit:]]+\./{sub(/\./,"\nnext line")} 1'
123
next line

Or use GNU sed which supports \n

Solution 2 - Bash

The portable way to get a newline in sed is a backslash followed by a literal newline:

$ echo 'foo' | sed 's/foo/foo\
bar/'
foo
bar

I guarantee there's a far simpler solution to your whole problem by using awk rather than sed though.

Solution 3 - Bash

Add a line after a match.

The sed command can add a new line after a pattern match is found. The "a" command to sed tells it to add a new line after a match is found.

> sed '/unix/ a "Add a new line"' file.txt

unix is great os. unix is opensource. unix is free os.

    "Add a new line"
    
    learn operating system.
    
    unixlinux which one you choose.
    
    "Add a new line"

Add a line before a match

The sed command can add a new line before a pattern match is found. The "i" command to sed tells it to add a new line before a match is found.

> sed '/unix/ i "Add a new line"' file.txt

"Add a new line"

unix is great os. unix is opensource. unix is free os.

learn operating system.

"Add a new line"

unixlinux which one you choose.

Solution 4 - Bash

The following works on Oracle Linux, x8664:

$ echo 'foobar' | sed 's/foo/foo\n/'
foo
bar

If you need it to match more than once per line, you'll need to place a g at the end, as in:

$ echo 'foobarfoobaz' | sed 's/foo/foo\n/g'
foo
barfoo
baz

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAllanView Question on Stackoverflow
Solution 1 - BashdawgView Answer on Stackoverflow
Solution 2 - BashEd MortonView Answer on Stackoverflow
Solution 3 - BashMohasin AliView Answer on Stackoverflow
Solution 4 - Bashinterestedparty333View Answer on Stackoverflow