How to remove a character at the end of each line in UNIX

UnixAwkSed

Unix Problem Overview


I would like to remove comma , at the end of each line in my file. How can I do it other than using substring function in awk?

Sample Input:

        SUPPLIER_PROC_ID BIGINT NOT NULL,
    	BTCH_NBR INTEGER NOT NULL,
    	RX_BTCH_SUPPLIER_SEQ_NBR INTEGER NOT NULL,
    	CORRN_ID INTEGER NOT NULL,
    	RX_CNT BYTEINT NOT NULL,
    	DATA_TYP_CD BYTEINT NOT NULL,
    	DATA_PD_CD BYTEINT NOT NULL,
    	CYC_DT DATE NOT NULL,
    	BASE_DT DATE NOT NULL,
    	DATA_LOAD_DT DATE NOT NULL,
    	DATA_DT DATE NOT NULL,
    	SUPPLIER_DATA_SRC_CD BYTEINT NOT NULL,
    	RX_CHNL_CD BYTEINT NOT NULL,
    	MP_IMS_ID INTEGER NOT NULL,
    	MP_LOC_ID NUMERIC(3,0),
    	MP_IMS_ID_ACTN_CD BYTEINT NOT NULL,
    	NPI_ID BIGINT,

Unix Solutions


Solution 1 - Unix

Try doing this :

awk '{print substr($0, 1, length($0)-1)}' file.txt

This is more generic than just removing the final comma but any last character

If you'd want to only remove the last comma with awk :

awk '{gsub(/,$/,""); print}' file.txt

Solution 2 - Unix

You can use sed:

sed 's/,$//' file > file.nocomma

and to remove whatever last character:

sed 's/.$//' file > file.nolast

Solution 3 - Unix

An awk code based on RS.

awk '1' RS=',\n' file

or:

awk 'BEGIN{RS=",\n"}1' file

This last example will be valid for any char before newline:

awk '1' RS='.\n' file

Note: dot . matches any character except line breaks.

Explanation

awk allows us to use different record (line) regex separators, we just need to include the comma before the line break (or dot for any char) in the one used for the input, the RS.

Note: what that 1 means?

Short answer, It's just a shortcut to avoid using the print statement. In awk when a condition gets matched the default action is to print the input line, example:

$ echo "test" |awk '1'
test

That's because 1 will be always true, so this expression is equivalent to:

$ echo "test"|awk '1==1'
test
$ echo "test"|awk '{if (1==1){print}}'
test

Documentation

Check Record Splitting with Standard awk and Output Separators.

Solution 4 - Unix

This Perl code removes commas at the end of the line:

perl -pe 's/,$//' file > file.nocomma

This variation still works if there is whitespace after the comma:

perl -lpe 's/,\s*$//' file > file.nocomma

This variation edits the file in-place:

perl -i -lpe 's/,\s*$//' file

This variation edits the file in-place, and makes a backup file.bak:

perl -i.bak -lpe 's/,\s*$//' file

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionTejaView Question on Stackoverflow
Solution 1 - UnixGilles QuenotView Answer on Stackoverflow
Solution 2 - UnixjlliagreView Answer on Stackoverflow
Solution 3 - UnixJuan Diego Godoy RoblesView Answer on Stackoverflow
Solution 4 - UnixChris KoknatView Answer on Stackoverflow