Get page output with curl --fail
HttpCurlHttp PostWgetPostfileHttp Problem Overview
Calling curl without parameters, i get the page output, even with an http status code = 404:
$ curl http://www.google.com/linux;
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 404 (Not Found)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/errors/logo_sm_2.png) no-repeat}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/errors/logo_sm_2_hr.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:55px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>404.</b> <ins>That’s an error.</ins>
<p>The requested URL <code>/linux</code> was not found on this server. <ins>That’s all we know.</ins>
$ echo $?;
0
The status code is 0.
Calling it with --fail will not show the output:
$ curl --fail http://www.google.com/linux;
curl: (22) The requested URL returned error: 404 Not Found
$ echo $?;
22
The status code is 22 now ...
Id' like to get the output even when http status = 404, 500 (like the first curl execution) and, at the same time, get a different system error (like in the second curl execution, $? = 22). Is it possible with curl? If not, how could I achieve this with another tool (this tool must accept file uploads e post data! wget doesn't seems to be an alternative ...)
Thanks.
Http Solutions
Solution 1 - Http
First of all the maximum value for the error code(or exit code) is 255. Here is the reference.
Also, the --fail will not allow you to do what you are looking for. However, you can use alternate ways(writing a shell script) to handle the scenario, but not sure it will be effective or not for you!
http_code=$(curl -s -o out.html -w '%{http_code}' http://www.google.com/linux;)
if [[ $http_code -eq 200 ]]; then
exit 0
fi
## decide which status you want to return for 404 or 500
exit 204
Now do the $? and you'll get the exit code from there.
You'll find the response html inside the out.html file.
You can also pass the url to the script as commandline argument. Check here.
Solution 2 - Http
Unfortunately not possible with curl. But you can do this with wget.
$ wget --content-on-error -qO- http://httpbin.org/status/418
-=[ teapot ]=-
_...._
.' _ _ `.
| ."` ^ `". _,
\_;`"---"`|//
| ;/
\_ _/
`"""`
$ echo $?
8
Solution 3 - Http
This is now possible with curl. Since version 7.76.0 you can do
curl --fail-with-body ...
Which does exactly what OP asked: shows the document body and exits with code 22.
Solution 4 - Http
I found a solution because wget was not suitable for sending multipart/form-data
curl -o - -w "\n%{http_code}\n" http://httpbin.org/status/418 | tee >(tail -n 1 | cmp <(echo 2xx) - ) | tee >(grep "char 2"; echo $? > status-code) && grep 0 status-code
Explanation
-o - -w "\n%{http_code}\n" - prints out to stdout (actually it's piped to the next command) with status code at the end
tee - output will be piped to next command and additionally printed to stdout
tail -n 1 - extract status code from the last line
cmp <(echo 2xx) - compare status code, first char only
grep "char 2" - if first character needs to be 2, otherwise fail
In a shell script you can also do better comparison (currently it only allows 2xx, so redirect like 300 are are handled as an error with cmp how it is used above)
Solution 5 - Http
Thanks @timaschew, here is my enhanced version based on pure awk:
curl_fail_with_body() {
curl -o - -w "\n%{http_code}\n" "$@" | awk '{l[NR] = $0} END {for (i=1; i<=NR-1; i++) print l[i]}; END{ if ($0<200||$0>299) exit $0 }'
}
# example usage
curl_fail_with_body -sS http://httpbin.org/status/418
Explanation
-o - -w "\n%{http_code}\n"- prints out to stdout (actually it's piped to the next command) with status code at the end{l[NR] = $0} END {for (i=1; i<=NR-1; i++) print l[i]}- print all the lines except the last oneEND{ if ($0<200||$0>299) exit $0 }- will exit with non zero code if thelast line != 2xx
alternative version, if you want to output the error code after command:
END{ if ($0<200||$0>299) {print "The requested URL returned error: " $0; exit 1}
BTW, curl supports --fail-with-body option since v7.76.0.
This option allows you to achieve the desired behavior without using external tools.
Solution 6 - Http
Here was my solution - it uses jq and assumes the body is json
# this code adds a statusCode field to the json it receives and then jq squeezes them together
# curl 7.76.0 will have curl --fail-with-body and thus eliminate all this
local result
result=$(
curl -sL -w ' { "statusCode": %{http_code}} ' -X POST "${headers[@]}" "${endpoint}" \
-d "${body}" "$curl_opts" | jq -ren '[inputs] | add'
)
# always output the result
echo "${result}"
# jq -e will produce an error code if the expression result is false or null - thus resulting in a
# error return code from this function naturally. This is much preferred rather than assume/hardcode
# the existence of a error object in the body payload
echo "${result}" | jq -re '.statusCode >= 200 and .statusCode < 300' > /dev/null