Assigning system command's output to variable

AwkPipe

Awk Problem Overview


I want to run the system command in an awk script and get its output stored in a variable. I've been trying to do this, but the command's output always goes to the shell and I'm not able to capture it. Any ideas on how this can be done?

Example:

$ date | awk --field-separator=! {$1 = system("strip $1"); /*more processing*/}

Should call the strip system command and instead of sending the output to the shell, should assign the output back to $1 for more processing. Rignt now, it's sending output to shell and assigning the command's retcode to $1.

Awk Solutions


Solution 1 - Awk

Note: Coprocess is GNU awk specific. Anyway another alternative is using getline

cmd = "strip "$1
while ( ( cmd | getline result ) > 0 ) {
  print  result
} 
close(cmd)

Calling close(cmd) will prevent awk to throw this error after a number of calls :

> fatal: cannot open pipe `…' (Too many open files)

Solution 2 - Awk

To run a system command in awk you can either use system() or cmd | getline.

I prefer cmd | getline because it allows you to catch the value into a variable:

$ awk 'BEGIN {"date" |  getline mydate; close("date"); print "returns", mydate}'
returns Thu Jul 28 10:16:55 CEST 2016

More generally, you can set the command into a variable:

awk 'BEGIN {
       cmd = "date -j -f %s"
       cmd | getline mydate
       close(cmd)
     }'

Note it is important to use close() to prevent getting a "makes too many open files" error if you have multiple results (thanks mateuscb for pointing this out in comments).


Using system(), the command output is printed automatically and the value you can catch is its return code:

$ awk 'BEGIN {d=system("date"); print "returns", d}'
Thu Jul 28 10:16:12 CEST 2016
returns 0
$ awk 'BEGIN {d=system("ls -l asdfasdfasd"); print "returns", d}'
ls: cannot access asdfasdfasd: No such file or directory
returns 2

Solution 3 - Awk

Figured out.

We use awk's Two-way I/O

{
  "strip $1" |& getline $1
}

passes $1 to strip and the getline takes output from strip back to $1

Solution 4 - Awk

gawk '{dt=substr($4,2,11); gsub(/\//," ",dt); "date -d \""dt"\" +%s"|getline ts; print ts}'

Solution 5 - Awk

You can use this when you need to process a grep output:

echo "some/path/exex.c:some text" | awk -F: '{ "basename "$1"" |& getline $1; print $1 " ==> " $2}'

option -F: tell awk to use : as field separator

"basename "$1"" execute shell command basename on first field

|& getline $1 reads output of previous shell command in substream

output:
exex.c ==> some text

Solution 6 - Awk

I am using macOS's awk and I also needed exit status of the command. So I extended @ghostdog74's solution to get the exit status too:

Exit if non-zero exit status:

cmd = <your command goes here>
cmd = cmd" ; printf \"\n$?\""

last_res = ""
value = ""        

while ( ( cmd | getline res ) > 0 ) {

    if (value == "") {
        value = last_res
    } else {
        value = value"\n"last_res
    }
        
    last_res = res
}

close(cmd)

# Now `res` has the exit status of the command
# and `value` has the complete output of command

if (res != 0) {
    exit 1
} else {
    print value
}

So basically I just changed cmd to print exit status of the command on a new line. After the execution of the above while loop, res would contain the exit status of the command and value would contain the complete output of the command.

Honestly not a very neat way and I myself would like to know if there is some better way.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionSahasView Question on Stackoverflow
Solution 1 - Awkghostdog74View Answer on Stackoverflow
Solution 2 - AwkfedorquiView Answer on Stackoverflow
Solution 3 - AwkSahasView Answer on Stackoverflow
Solution 4 - AwkRyan LiuView Answer on Stackoverflow
Solution 5 - AwkDmitryView Answer on Stackoverflow
Solution 6 - AwkMihir LuthraView Answer on Stackoverflow