Can PHP cURL retrieve response headers AND body in a single request?

PhpHttpCurl

Php Problem Overview


Is there any way to get both headers and body for a cURL request using PHP? I found that this option:

curl_setopt($ch, CURLOPT_HEADER, true);

is going to return the body plus headers, but then I need to parse it to get the body. Is there any way to get both in a more usable (and secure) way?

Note that for "single request" I mean avoiding issuing a HEAD request prior of GET/POST.

Php Solutions


Solution 1 - Php

One solution to this was posted in the PHP documentation comments: http://www.php.net/manual/en/function.curl-exec.php#80442

Code example:

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// ...

$response = curl_exec($ch);

// Then, after your curl_exec call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);

Warning: As noted in the comments below, this may not be reliable when used with proxy servers or when handling certain types of redirects. @Geoffrey's answer may handle these more reliably.

Solution 2 - Php

Many of the other solutions offered this thread are not doing this correctly.

  • Splitting on \r\n\r\n is not reliable when CURLOPT_FOLLOWLOCATION is on or when the server responds with a 100 code.
  • Not all servers are standards compliant and transmit just a \n for new lines.
  • Detecting the size of the headers via CURLINFO_HEADER_SIZE is also not always reliable, especially when proxies are used or in some of the same redirection scenarios.

The most correct method is using CURLOPT_HEADERFUNCTION.

Here is a very clean method of performing this using PHP closures. It also converts all headers to lowercase for consistent handling across servers and HTTP versions.

This version will retain duplicated headers

This complies with RFC822 and RFC2616, please do not suggest edits to make use of the mb_ string functions, it is incorrect!

$ch = curl_init();
$headers = [];
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// this function is called by curl for each header received
curl_setopt($ch, CURLOPT_HEADERFUNCTION,
  function($curl, $header) use (&$headers)
  {
    $len = strlen($header);
    $header = explode(':', $header, 2);
    if (count($header) < 2) // ignore invalid headers
      return $len;

    $headers[strtolower(trim($header[0]))][] = trim($header[1]);
    
    return $len;
  }
);

$data = curl_exec($ch);
print_r($headers);

Solution 3 - Php

Curl has a built in option for this, called CURLOPT_HEADERFUNCTION. The value of this option must be the name of a callback function. Curl will pass the header (and the header only!) to this callback function, line-by-line (so the function will be called for each header line, starting from the top of the header section). Your callback function then can do anything with it (and must return the number of bytes of the given line). Here is a tested working code:

function HandleHeaderLine( $curl, $header_line ) {
    echo "<br>YEAH: ".$header_line; // or do whatever
    return strlen($header_line);
}


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.google.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, "HandleHeaderLine");
$body = curl_exec($ch); 

The above works with everything, different protocols and proxies too, and you dont need to worry about the header size, or set lots of different curl options.

P.S.: To handle the header lines with an object method, do this:

curl_setopt($ch, CURLOPT_HEADERFUNCTION, array($object, 'methodName'))

Solution 4 - Php

is this what are you looking to?

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
$response = curl_exec($ch); 
list($header, $body) = explode("\r\n\r\n", $response, 2);

Solution 5 - Php

If you specifically want the Content-Type, there's a special cURL option to retrieve it:

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
$content_type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE);

Solution 6 - Php

Just set options :

  • CURLOPT_HEADER, 0

  • CURLOPT_RETURNTRANSFER, 1

and use curl_getinfo with CURLINFO_HTTP_CODE (or no opt param and you will have an associative array with all the informations you want)

More at : http://php.net/manual/fr/function.curl-getinfo.php

Solution 7 - Php

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);

$parts = explode("\r\n\r\nHTTP/", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = explode("\r\n\r\n", $parts, 2);

Works with HTTP/1.1 100 Continue before other headers.

If you need work with buggy servers which sends only LF instead of CRLF as line breaks you can use preg_split as follows:

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);

$parts = preg_split("@\r?\n\r?\nHTTP/@u", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = preg_split("@\r?\n\r?\n@u", $parts, 2);

Solution 8 - Php

My way is

$response = curl_exec($ch);
$x = explode("\r\n\r\n", $v, 3);
$header=http_parse_headers($x[0]);
if ($header=['Response Code']==100){ //use the other "header"
    $header=http_parse_headers($x[1]);
    $body=$x[2];
}else{
    $body=$x[1];
}

If needed apply a for loop and remove the explode limit.

Solution 9 - Php

Here is my contribution to the debate ... This returns a single array with the data separated and the headers listed. This works on the basis that CURL will return a headers chunk [ blank line ] data

curl_setopt($ch, CURLOPT_HEADER, 1); // we need this to get headers back
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, true);

// $output contains the output string
$output = curl_exec($ch);

$lines = explode("\n",$output);

$out = array();
$headers = true;

foreach ($lines as $l){
	$l = trim($l);

	if ($headers && !empty($l)){
		if (strpos($l,'HTTP') !== false){
			$p = explode(' ',$l);
			$out['Headers']['Status'] = trim($p[1]);
		} else {
			$p = explode(':',$l);
			$out['Headers'][$p[0]] = trim($p[1]);
		}
	} elseif (!empty($l)) {
		$out['Data'] = $l;
	}

	if (empty($l)){
		$headers = false;
	}
}

Solution 10 - Php

The problem with many answers here is that "\r\n\r\n" can legitimately appear in the body of the html, so you can't be sure that you're splitting headers correctly.

It seems that the only way to store headers separately with one call to curl_exec is to use a callback as is suggested above in https://stackoverflow.com/a/25118032/3326494

And then to (reliably) get just the body of the request, you would need to pass the value of the Content-Length header to substr() as a negative start value.

Solution 11 - Php

A better way is to use the verbose CURL response which can be piped to a temporary stream. Then you can search the response for the header name. This could probably use a few tweaks but it works for me:

class genericCURL {
	/**
	 * NB this is designed for getting data, or for posting JSON data
	 */
	public function request($url, $method = 'GET', $data = array()) {
		$ch = curl_init();
		
		if($method == 'POST') {
			
			curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
			curl_setopt($ch, CURLOPT_POSTFIELDS, $string = json_encode($data));
			
		}

		
		curl_setopt($ch, CURLOPT_URL, $url);
		curl_setopt($ch, CURLOPT_VERBOSE, true);
		
		//open a temporary stream to output the curl log, which would normally got to STDERR
		$err = fopen("php://temp", "w+");
		curl_setopt($ch, CURLOPT_STDERR, $err);
		

		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
		$server_output = curl_exec ($ch);
		
		//rewind the temp stream and put it into a string	
		rewind($err);
		$this->curl_log = stream_get_contents($err);
		
		curl_close($ch);
		fclose($err);

	
		return $server_output;
		
	}
	
	/**
	 * use the curl log to get a header value
	 */
	public function getReturnHeaderValue($header) {
		$log = explode("\n", str_replace("\r\n", "\n", $this->curl_log));
		foreach($log as $line) {
			//is the requested header there
			if(stripos($line, '< ' . $header . ':') !== false) {
				$value = trim(substr($line, strlen($header) + 3));
				return $value;
			}
		}
		//still here implies not found so return false
		return false;
		
	}
}

Solution 12 - Php

Improvement of Geoffreys answer:

I couldn't get the right length for header with $headerSize = curl_getinfo($this->curlHandler, CURLINFO_HEADER_SIZE);- i had to calculate header size on my own.

In addition some improvements for better readability.

$headerSize = 0;
curl_setopt_array($this->curlHandler, [
CURLOPT_URL => $yourUrl,
CURLOPT_POST => 0,
CURLOPT_RETURNTRANSFER => 1,
// this function is called by curl for each header received
CURLOPT_HEADERFUNCTION =>
         function ($curl, $header) use (&$headers, &$headerSize) {
              $lenghtCurrentLine = strlen($header);
              $headerSize += $lenghtCurrentLine;
              $header = explode(':', $header, 2);
              if (count($header) > 1) { // store only vadid headers
                   $headers[strtolower(trim($header[0]))][] = trim($header[1]);
              }
              return $lenghtCurrentLine;
           },
]);
$fullResult = curl_exec($this->curlHandler);
$result = substr($fullResult, $headerSize);

Solution 13 - Php

Just in case you can't / don't use CURLOPT_HEADERFUNCTION or other solutions;

$nextCheck = function($body) {
    return ($body && strpos($body, 'HTTP/') === 0);
};

[$headers, $body] = explode("\r\n\r\n", $result, 2);
if ($nextCheck($body)) {
    do {
        [$headers, $body] = explode("\r\n\r\n", $body, 2);
    } while ($nextCheck($body));
}

Solution 14 - Php

Try this if you are using GET:

$curl = curl_init($url);

curl_setopt_array($curl, array(
    CURLOPT_URL => $url,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET",
    CURLOPT_HTTPHEADER => array(
        "Cache-Control: no-cache"
    ),
));

$response = curl_exec($curl);
curl_close($curl);

Solution 15 - Php

Return response headers with a reference parameter:

<?php
$data=array('device_token'=>'5641c5b10751c49c07ceb4',
            'content'=>'测试测试test'
           );
$rtn=curl_to_host('POST', 'http://test.com/send_by_device_token', array(), $data, $resp_headers);
echo $rtn;
var_export($resp_headers);

function curl_to_host($method, $url, $headers, $data, &$resp_headers)
         {$ch=curl_init($url);
          curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $GLOBALS['POST_TO_HOST.LINE_TIMEOUT']?$GLOBALS['POST_TO_HOST.LINE_TIMEOUT']:5);
          curl_setopt($ch, CURLOPT_TIMEOUT, $GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']?$GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']:20);
          curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
          curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
          curl_setopt($ch, CURLOPT_HEADER, 1);

          if ($method=='POST')
             {curl_setopt($ch, CURLOPT_POST, true);
              curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
             }
          foreach ($headers as $k=>$v)
                  {$headers[$k]=str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', $k)))).': '.$v;
                  }
          curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
          $rtn=curl_exec($ch);
          curl_close($ch);

          $rtn=explode("\r\n\r\nHTTP/", $rtn, 2);    //to deal with "HTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 200 OK...\r\n\r\n..." header
          $rtn=(count($rtn)>1 ? 'HTTP/' : '').array_pop($rtn);
          list($str_resp_headers, $rtn)=explode("\r\n\r\n", $rtn, 2);

          $str_resp_headers=explode("\r\n", $str_resp_headers);
          array_shift($str_resp_headers);    //get rid of "HTTP/1.1 200 OK"
          $resp_headers=array();
          foreach ($str_resp_headers as $k=>$v)
                  {$v=explode(': ', $v, 2);
                   $resp_headers[$v[0]]=$v[1];
                  }

          return $rtn;
         }
?>

Solution 16 - Php

If you don't really need to use curl;

$body = file_get_contents('http://example.com');
var_export($http_response_header);
var_export($body);

Which outputs

array (
  0 => 'HTTP/1.0 200 OK',
  1 => 'Accept-Ranges: bytes',
  2 => 'Cache-Control: max-age=604800',
  3 => 'Content-Type: text/html',
  4 => 'Date: Tue, 24 Feb 2015 20:37:13 GMT',
  5 => 'Etag: "359670651"',
  6 => 'Expires: Tue, 03 Mar 2015 20:37:13 GMT',
  7 => 'Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT',
  8 => 'Server: ECS (cpm/F9D5)',
  9 => 'X-Cache: HIT',
  10 => 'x-ec-custom-error: 1',
  11 => 'Content-Length: 1270',
  12 => 'Connection: close',
)'<!doctype html>
<html>
<head>
    <title>Example Domain</title>...

See http://php.net/manual/en/reserved.variables.httpresponseheader.php

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiongremoView Question on Stackoverflow
Solution 1 - PhpiblueView Answer on Stackoverflow
Solution 2 - PhpGeoffreyView Answer on Stackoverflow
Solution 3 - PhpSkaccView Answer on Stackoverflow
Solution 4 - Phpuser1031143View Answer on Stackoverflow
Solution 5 - Phppr1001View Answer on Stackoverflow
Solution 6 - PhpCyril H.View Answer on Stackoverflow
Solution 7 - PhpEnybyView Answer on Stackoverflow
Solution 8 - PhpRoyView Answer on Stackoverflow
Solution 9 - PhpAntonyView Answer on Stackoverflow
Solution 10 - PhpmalView Answer on Stackoverflow
Solution 11 - Phpjames-geldartView Answer on Stackoverflow
Solution 12 - PhpsneakyView Answer on Stackoverflow
Solution 13 - PhpK-GunView Answer on Stackoverflow
Solution 14 - PhptosturawView Answer on Stackoverflow
Solution 15 - PhpdiyismView Answer on Stackoverflow
Solution 16 - PhpBevanView Answer on Stackoverflow