What are NR and FNR and what does "NR==FNR" imply?

Awk

Awk Problem Overview


I am learning file comparison using awk.

I found syntax like below,

awk 'NR==FNR{a[$1];next}$1 in a{print $1}' file1 file2

I couldn't understand what is the significance of NR==FNR in this? If I try with FNR==NR then also I get the same output?

What exactly does it do?

Awk Solutions


Solution 1 - Awk

In awk,

FNR refers to the record number (typically the line number) in the current file,
NR refers to the total record number.
The operator == is a comparison operator, which returns true when the two surrounding operands are equal.

This means that the condition NR==FNR is only true for the first file, as FNR resets back to 1 for the first line of each file but NR keeps on increasing.
This pattern is typically used to perform actions on only the first file.

The next inside the block means any further commands are skipped, so they are only run on files other than the first.

The condition FNR==NR compares the same two operands as NR==FNR, so it behaves in the same way.

Solution 2 - Awk

Look for keys (first word of line) in file2 that are also in file1.
Step 1: fill array a with the first words of file 1:

awk '{a[$1];}' file1

Step 2: Fill array a and ignore file 2 in the same command. For this check the total number of records until now with the number of the current input file.

awk 'NR==FNR{a[$1]}' file1 file2

Step 3: Ignore actions that might come after } when parsing file 1

awk 'NR==FNR{a[$1];next}' file1 file2 

Step 4: print key of file2 when found in the array a

awk 'NR==FNR{a[$1];next} $1 in a{print $1}' file1 file2

Solution 3 - Awk

Look up NR and FNR in the awk manual and then ask yourself what is the condition under which NR==FNR in the following example:

$ cat file1
a
b
c

$ cat file2
d
e

$ awk '{print FILENAME, NR, FNR, $0}' file1 file2
file1 1 1 a
file1 2 2 b
file1 3 3 c
file2 4 1 d
file2 5 2 e

Solution 4 - Awk

There are awk built-in variables.

NR - It gives the total number of records processed.

FNR - It gives the total number of records for each input file.

Solution 5 - Awk

Assuming you have Files a.txt and b.txt with

cat a.txt
a
b
c
d
1
3
5
cat b.txt
a
1
2
6
7

Keep in mind NR and FNR are awk built-in variables. NR - Gives the total number of records processed. (in this case both in a.txt and b.txt) FNR - Gives the total number of records for each input file (records in either a.txt or b.txt)

awk 'NR==FNR{a[$0];}{if($0 in a)print FILENAME " " NR " " FNR " " $0}' a.txt b.txt
a.txt 1 1 a
a.txt 2 2 b
a.txt 3 3 c
a.txt 4 4 d
a.txt 5 5 1
a.txt 6 6 3
a.txt 7 7 5
b.txt 8 1 a
b.txt 9 2 1

lets Add "next" to skip the first matched with NR==FNR

in b.txt and in a.txt

awk 'NR==FNR{a[$0];next}{if($0 in a)print FILENAME " " NR " " FNR " " $0}' a.txt b.txt
b.txt 8 1 a
b.txt 9 2 1

in b.txt but not in a.txt

 awk 'NR==FNR{a[$0];next}{if(!($0 in a))print FILENAME " " NR " " FNR " " $0}' a.txt b.txt
b.txt 10 3 2
b.txt 11 4 6
b.txt 12 5 7

awk 'NR==FNR{a[$0];next}!($0 in a)' a.txt b.txt
2
6
7

Solution 6 - Awk

Here is the pseudo code for your interest.

NR = 1
for (i=1; i<=files.length; ++i) {
    line = read line from files[i]
    FNR = 1
    while (not EOF) {
        columns = getColumns(line)

        if (NR is equals to FNR) { // processing first file
            add columns[1] to a
        } else { // processing remaining files
            if (columns[1] exists in a) {
                print columns[1]
            }
        }
        NR = NR + 1
        FNR = FNR + 1
        line = read line from files[i]
    }
}

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAmitView Question on Stackoverflow
Solution 1 - AwkTom FenechView Answer on Stackoverflow
Solution 2 - AwkWalter AView Answer on Stackoverflow
Solution 3 - AwkEd MortonView Answer on Stackoverflow
Solution 4 - AwksatView Answer on Stackoverflow
Solution 5 - AwkDon Kepler Brian SerembaView Answer on Stackoverflow
Solution 6 - AwkFranz WongView Answer on Stackoverflow