use pipe for curl data

CurlPipe

Curl Problem Overview


I'm trying to pass the cat output to curl:

$ cat file | curl --data '{"title":"mytitle","input":"-"}' http://api

But input is literally a -.

Curl Solutions


Solution 1 - Curl

I spent a while trying to figure this out and got it working with the following:

cat data.json | curl -H "Content-Type: application/json" -X POST --data-binary @- http://api

Solution 2 - Curl

You can use the magical stdin file /dev/stdin

cat data.json | curl -H "Content-Type: application/json" -X POST -d "$(</dev/stdin)" http://api

Solution 3 - Curl

This should also work

curl -H "Content-Type: application/json" -d @data.json http://api

Using -d forces curl to implicitly use POST for the request.

Solution 4 - Curl

# Create the input file
echo -n 'Try 😁 and " to verify proper JSON encoding.' > file.txt

# 1. Use jq to read the file into variable named `input` 
# 2. create the desired json
# 3. pipe the result into curl
jq -n --rawfile input file.txt '{"title":"mytitle", $input}' \
| curl -v 'https://httpbin.org/post' -H 'Content-Type: application/json' -d@- 

Output:

  ...
  "json": {
    "input": "Try \ud83d\ude01 and \" to verify proper JSON encoding.", 
    "title": "mytitle"
  }, 
  ...

Notice that the contents of the input file was properly escaped for using as a JSON value.

jq options used:

  • --null-input/-n:
    Don’t read any input
  • --rawfile variable-name filename:
    This option reads in the named file and binds its contents to the given global variable.

See jq manual for full details.

The -d@- option tells curl to read the data from STDIN.

Solution 5 - Curl

If you want to type/paste the data without escaping or polluting your bash history then, you can use this

cat | curl -H 'Content-Type: application/json' http://api -d @-

Which drops you into cat where you can input the data, directly, e.g. Shift + Insert in your terminal. You finish with a newline and a Ctrl + D which signals to cat that you're done. That data is then passed to curl, and you have a reusable history entry.

Solution 6 - Curl

Try

curl --data '{"title":"mytitle","input":"'$(cat file)'-"}' http://api

Solution 7 - Curl

Curl documentation for -d option >If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with -d, --data @foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out. If you don't want the @ character to have a special interpretation use --data-raw instead.

Depending of your HTTP endpoint, server configuration, you should be good by using this format:

curl -d @data.json http://api

Solution 8 - Curl

Sounds like you want to wrap the content of input in a JSON body, and then have that sent over with a POST request. I think that the simplest way to do that is to manipulate stdin first and then push that over to curl using -d @-. One way could look like this:

cat <(echo '{"title":"mytitle","input":"') file <(echo '"}') \
| curl -d @- http://api

I'm using <(echo) to use cat to merge strings and files, but there is almost certainly a better way.

Keep in mind that this does not escape the contents of file and that you may run into issues because of that.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJ&#252;rgen PaulView Question on Stackoverflow
Solution 1 - CurlTom JowittView Answer on Stackoverflow
Solution 2 - CurlNathanView Answer on Stackoverflow
Solution 3 - CurlJoseph KingView Answer on Stackoverflow
Solution 4 - CurlRobin A. MeadeView Answer on Stackoverflow
Solution 5 - CurlWalfView Answer on Stackoverflow
Solution 6 - CurlgolimarView Answer on Stackoverflow
Solution 7 - CurlricherlariviereView Answer on Stackoverflow
Solution 8 - CurlzneakView Answer on Stackoverflow