find difference between two text files with one item per line
BashFileScriptingSedAwkBash Problem Overview
I have two files:
file 1
dsf
sdfsd
dsfsdf
file 2
ljljlj
lkklk
dsf
sdfsd
dsfsdf
I want to display what is in file 2 but not in file 1, so file 3 should look like
ljljlj
lkklk
Bash Solutions
Solution 1 - Bash
grep -Fxvf file1 file2
What the flags mean:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.
-x, --line-regexp
Select only those matches that exactly match the whole line.
-v, --invert-match
Invert the sense of matching, to select non-matching lines.
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file contains zero patterns, and therefore matches nothing.
Solution 2 - Bash
You can try
grep -f file1 file2
or
grep -v -F -x -f file1 file2
Solution 3 - Bash
You can use the comm command to compare two sorted files
comm -13 <(sort file1) <(sort file2)
Solution 4 - Bash
I successfully used
diff "${file1}" "${file2}" | grep "<" | sed 's/^<//g' > "${diff_file}"
Outputting the difference to a file.
Solution 5 - Bash
if you are expecting them in a certain order, you can just use diff
diff file1 file2 | grep ">"
Solution 6 - Bash
join -v 2 <(sort file1) <(sort file2)
Solution 7 - Bash
A tried a slight variation on Luca's answer and it worked for me.
diff file1 file2 | grep ">" | sed 's/^> //g' > diff_file
Note that the searched pattern in sed is a > followed by a space.
Solution 8 - Bash
file1 m1 m2 m3file2 m2 m4 m5
>awk 'NR == FNR {file1[$0]++; next} !($0 in file1)' file1 file2 m4 m5
>awk 'NR == FNR {file1[$0]++; next} ($0 in file1)' file1 file2 m2
> What's awk command to get 'm1 and m3' ?? as in file1 and not in file2? m1 m3
Solution 9 - Bash
If you want to use loops You can try like this: (diff and cmp are much more efficient. )
while read line
do
flag = 0
while read line2
do
if ( "$line" = "$line2" )
then
flag = 1
fi
done < file1
if ( flag -eq 0 )
then
echo $line > file3
fi
done < file2
Note: The program is only to provide a basic insight into what can be done if u dont want to use system calls such as diff n comm..
Solution 10 - Bash
an awk answer:
awk 'NR == FNR {file1[$0]++; next} !($0 in file1)' file1 file2
Solution 11 - Bash
With GNU sed:
sed 's#[^^]#[&]#g;s#\^#\\^#g;s#^#/^#;s#$#$/d#' file1 | sed -f- file2
How it works:
The first sed produces an output like this:
/^[d][s][f]$/d
/^[s][d][f][s][d]$/d
/^[d][s][f][s][d][f]$/d
Then it is used as a sed script by the second sed.