curl: read headers from file

Curl

Curl Problem Overview


After the --dump-header writes a file, how to read those headers back into the next request? I would like to read them from a file because there are a number of them.

I tried standard in: cat headers | curl -v -H - ...

I'm actually using the feature in Firebug to "Copy Request Headers" and then saving those to a file. This appears to be the same format.

Curl Solutions


Solution 1 - Curl

since curl 7.55.0

Easy:

$ curl -H @header_file $URL

... where the header file is a plain text file with a HTTP header on each line. Like this:

Color: red
Shoesize: 11
Secret: yes
User-Agent: foobar/3000
Name: "Joe Smith"

before curl 7.55.0

curl had no way to "bulk change" headers like that, not even from a file.

Your best approach with an old curl version is probably to instead write a shell script that gathers all the headers from the file and use them, like:

#!/bin/sh
while read line; do
  args="$args -H '$line'";
done
curl $args $URL

Invoke the script like this:

$ sh script.sh < header_file

Solution 2 - Curl

how about this:

curl -v -H "$(cat headers.txt)" yourhost.com

where headers.txt looks like

Header1: bla
Header2: blupp

works in BASH.

Solution 3 - Curl

Starting with curl 7.55.0 it can now read headers from a file:

curl -H @filename

It's that easy now.

Solution 4 - Curl

As answered by @dmitry-sutyagin, if your curl is at least version 7.55.0 you can use the @ notation to read headers from a file:

curl -H @headerfile.txt https://www.google.com/  # requires curl 7.55.0

If your curl is NOT 7.55.0 or newer, there's a useful hack:

  • Use the option -K/--config <config file>, and put several -H/--header <header> lines in the text file.

For instance:

  1. curl --dump-header foo.txt https://www.google.com/

  2. If necessary, dos2unix foo.txt

  3. Convert the file to -H 'header' lines, manually or with a script:

    cat foo.txt |
      awk '$1 == "Set-Cookie:"' |
      perl -ne "chomp; next if /^\\s*\$/; if (/'/) { warn; next } print \"-H '\$_'\\n\";" |
      tee headerfile.txt
    

    This might output something like:

    -H 'Set-Cookie: 1P_JAR=2018-02-13-08; [...]'
    -H 'Set-Cookie: NID=123=n7vY1W8IDElvf [...]'
    
  4. curl --config headerfile.txt https://www.google.com/

Solution 5 - Curl

curl $(xargs -a headers.txt printf "-H '%s'") example.org

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionjcalfee314View Question on Stackoverflow
Solution 1 - CurlDaniel StenbergView Answer on Stackoverflow
Solution 2 - CurlCpt. SenkfussView Answer on Stackoverflow
Solution 3 - CurlDmitrii SutiaginView Answer on Stackoverflow
Solution 4 - CurltraalView Answer on Stackoverflow
Solution 5 - Curluser2683246View Answer on Stackoverflow