curl: read headers from file
CurlCurl Problem Overview
After the --dump-header writes a file, how to read those headers back into the next request? I would like to read them from a file because there are a number of them.
I tried standard in: cat headers | curl -v -H - ...
I'm actually using the feature in Firebug to "Copy Request Headers" and then saving those to a file. This appears to be the same format.
Curl Solutions
Solution 1 - Curl
since curl 7.55.0
Easy:
$ curl -H @header_file $URL
... where the header file is a plain text file with a HTTP header on each line. Like this:
Color: red
Shoesize: 11
Secret: yes
User-Agent: foobar/3000
Name: "Joe Smith"
before curl 7.55.0
curl had no way to "bulk change" headers like that, not even from a file.
Your best approach with an old curl version is probably to instead write a shell script that gathers all the headers from the file and use them, like:
#!/bin/sh
while read line; do
args="$args -H '$line'";
done
curl $args $URL
Invoke the script like this:
$ sh script.sh < header_file
Solution 2 - Curl
how about this:
curl -v -H "$(cat headers.txt)" yourhost.com
where headers.txt
looks like
Header1: bla
Header2: blupp
works in BASH.
Solution 3 - Curl
Starting with curl 7.55.0 it can now read headers from a file:
curl -H @filename
It's that easy now.
Solution 4 - Curl
As answered by @dmitry-sutyagin, if your curl is at least version 7.55.0 you can use the @
notation to read headers from a file:
curl -H @headerfile.txt https://www.google.com/ # requires curl 7.55.0
If your curl is NOT 7.55.0 or newer, there's a useful hack:
- Use the option
-K/--config <config file>
, and put several-H/--header <header>
lines in the text file.
For instance:
-
curl --dump-header foo.txt https://www.google.com/
-
If necessary,
dos2unix foo.txt
-
Convert the file to
-H 'header'
lines, manually or with a script:cat foo.txt | awk '$1 == "Set-Cookie:"' | perl -ne "chomp; next if /^\\s*\$/; if (/'/) { warn; next } print \"-H '\$_'\\n\";" | tee headerfile.txt
This might output something like:
-H 'Set-Cookie: 1P_JAR=2018-02-13-08; [...]' -H 'Set-Cookie: NID=123=n7vY1W8IDElvf [...]'
-
curl --config headerfile.txt https://www.google.com/
Solution 5 - Curl
curl $(xargs -a headers.txt printf "-H '%s'") example.org