How to remove a character at the end of each line in UNIX
UnixAwkSedUnix Problem Overview
I would like to remove comma ,
at the end of each line in my file. How can I do it other than using substring function in awk
?
Sample Input:
SUPPLIER_PROC_ID BIGINT NOT NULL,
BTCH_NBR INTEGER NOT NULL,
RX_BTCH_SUPPLIER_SEQ_NBR INTEGER NOT NULL,
CORRN_ID INTEGER NOT NULL,
RX_CNT BYTEINT NOT NULL,
DATA_TYP_CD BYTEINT NOT NULL,
DATA_PD_CD BYTEINT NOT NULL,
CYC_DT DATE NOT NULL,
BASE_DT DATE NOT NULL,
DATA_LOAD_DT DATE NOT NULL,
DATA_DT DATE NOT NULL,
SUPPLIER_DATA_SRC_CD BYTEINT NOT NULL,
RX_CHNL_CD BYTEINT NOT NULL,
MP_IMS_ID INTEGER NOT NULL,
MP_LOC_ID NUMERIC(3,0),
MP_IMS_ID_ACTN_CD BYTEINT NOT NULL,
NPI_ID BIGINT,
Unix Solutions
Solution 1 - Unix
Try doing this :
awk '{print substr($0, 1, length($0)-1)}' file.txt
This is more generic than just removing the final comma but any last character
If you'd want to only remove the last comma with awk :
awk '{gsub(/,$/,""); print}' file.txt
Solution 2 - Unix
You can use sed:
sed 's/,$//' file > file.nocomma
and to remove whatever last character:
sed 's/.$//' file > file.nolast
Solution 3 - Unix
An awk
code based on RS
.
awk '1' RS=',\n' file
or:
awk 'BEGIN{RS=",\n"}1' file
This last example will be valid for any char before newline:
awk '1' RS='.\n' file
Note: dot .
matches any character except line breaks.
Explanation
awk
allows us to use different record (line) regex
separators, we just need to include the comma before the line break (or dot
for any char) in the one used for the input
, the RS
.
Note: what that 1
means?
Short answer, It's just a shortcut to avoid using the print
statement.
In awk
when a condition gets matched the default action is to print the input line, example:
$ echo "test" |awk '1'
test
That's because 1
will be always true, so this expression is equivalent to:
$ echo "test"|awk '1==1'
test
$ echo "test"|awk '{if (1==1){print}}'
test
Documentation
Check Record Splitting with Standard awk and Output Separators.
Solution 4 - Unix
This Perl code removes commas at the end of the line:
perl -pe 's/,$//' file > file.nocomma
This variation still works if there is whitespace after the comma:
perl -lpe 's/,\s*$//' file > file.nocomma
This variation edits the file in-place:
perl -i -lpe 's/,\s*$//' file
This variation edits the file in-place, and makes a backup file.bak
:
perl -i.bak -lpe 's/,\s*$//' file