How can I pass variables from awk to a shell command?

ShellAwkParameter Passing

Shell Problem Overview


I am trying to run a shell command from within awk for each line of a file, and the shell command needs one input argument. I tried to use system(), but it didn't recognize the input argument.

Each line of this file is an address of a file, and I want to run a command to process that file. So, for a simple example I want to use 'wc' command for each line and pass $1 to wc.

awk '{system("wc $1")}' myfile

Shell Solutions


Solution 1 - Shell

you are close. you have to concatenate the command line with awk variables:

awk '{system("wc "$1)}' myfile

Solution 2 - Shell

You cannot grab the output of an awk system() call, you can only get the exit status. Use the getline/pipe or getline/variable/pipe constructs

awk '{
    cmd = "your_command " $1
    while (cmd | getline line) {
        do_something_with(line) 
    }
    close(cmd)
}' file

Solution 3 - Shell

FYI here's how to use awk to process files whose names are stored in a file (providing wc-like functionality in this example):

gawk '
NR==FNR { ARGV[ARGC++]=$0; next }
{ nW+=NF; nC+=(length($0) + 1) }
ENDFILE { print FILENAME, FNR, nW, nC; nW=nC=0 }
' file

The above uses GNU awk for ENDFILE. With other awks just store the values in an array and print in a loop in the END section.

Solution 4 - Shell

I would suggest another solution:

awk '{print $1}' myfile | xargs wc

the difference is that it executes wc once with multiple arguments. It often works (for example, with kill command)

Solution 5 - Shell

Or use the pipe | as in bash then retrive the output in a variable with awk's getline, like this

 zcat /var/log/fail2ban.log* | gawk  '/.*Ban.*/  {print $7};' | sort | uniq -c | sort | gawk '{ "geoiplookup " $2 "| cut -f2 -d: " | getline geoip; print $2 "\t\t" $1 " " geoip}'

That line will print all the banned IPs from your server along with their origin (country) using the geoip-bin package.

The last part of that one-liner is the one that affects us :

gawk '{ "geoiplookup " $2 "| cut -f2 -d: " | getline geoip; print $2 "\t\t" $1 " " geoip}'

It simply says : run the command "geoiplookup 182.193.192.4 | -f2 -d:" ($2 gets substituted as you may guess) and put the result of that command in geoip (the | getline geoip bit). Next, print something something and anything inside the geoip variable.

The complete example and the results can be found here, an article I wrote.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionVahid MirjaliliView Question on Stackoverflow
Solution 1 - ShellKentView Answer on Stackoverflow
Solution 2 - Shellglenn jackmanView Answer on Stackoverflow
Solution 3 - ShellEd MortonView Answer on Stackoverflow
Solution 4 - ShellConstantine KurbatovView Answer on Stackoverflow
Solution 5 - ShellychaoucheView Answer on Stackoverflow