Set cURL to use local virtual hosts
CurlHostsCurl Problem Overview
Using Apache or Ngnix I always create development sites based on real projects such as http://project1.loc
which, after adding to my .hosts
file, the browser has no problem using.
However, when I attempt to make a cURL request (http://project1.loc/post.json
) to that same URL I never get anything but a timeout. I'm assuming cURL doesn't care about my custom hosts and goes straight to a name server for it's info.
How can I fix this?
UPDATE I set a custom header "HOST: http://project1.loc" and now I am getting 400 errors - but they are instantaneous so I'm assuming that cURL is at least using the hosts file...
Curl Solutions
Solution 1 - Curl
Actually, curl has an option explicitly for this: --resolve
Instead of curl -H 'Host: yada.com' http://127.0.0.1/something
use curl --resolve 'yada.com:80:127.0.0.1' http://yada.com/something
What's the difference, you ask?
Among others, this works with HTTPS. Assuming your local server has a certificate for yada.com
, the first example above will fail because the yada.com
certificate doesn't match the 127.0.0.1
hostname in the URL.
The second example works correctly with HTTPS.
In essence, passing a "Host" header via -H
does hack your Host into the header set, but bypasses all of curl's host-specific intelligence. Using --resolve
leverages all of the normal logic that applies, but simply pretends the DNS lookup returned the data in your command-line option. It works just like /etc/hosts
should.
Note --resolve
takes a port number, so for HTTPS you would use
curl --resolve 'yada.com:443:127.0.0.1' https://yada.com/something
Solution 2 - Curl
EDIT: While this is currently accepted answer, readers might find this other answer by user John Hart more adapted to their needs. It uses an option which, according to user Ken, was introduced in version 7.21.3 (which was released in December 2010, i.e. after this initial answer).
In your edited question, you're using the URL as the host name, whereas it needs to be the host name only.
Try:
curl -H 'Host: project1.loc' http://127.0.0.1/something
where project1.loc
is just the host name and 127.0.0.1
is the target IP address.
(If you're using curl from a library and not on the command line, make sure you don't put http://
in the Host
header.)
Solution 3 - Curl
Either use a real fully qualified domain name (like dev.yourdomain.com
) that pointing to 127.0.0.1
or try editing the proper hosts file (usually /etc/hosts in *nix environments).
Solution 4 - Curl
It seems that this is not an uncommon problem.
Check this first.
If that doesn't help, you can install a local DNS server on Windows, such as this. Configure Windows to use localhost as the DNS server. This server can be configured to be authoritative for whatever fake domains you need, and to forward requests on to the real DNS servers for all other requests.
I personally think this is a bit over the top, and can't see why the hosts file wouldn't work. But it should solve the problem you're having. Make sure you set up your normal DNS servers as forwarders as well.
Solution 5 - Curl
Does the server actually get the requests, and are you handling the host name (alias) properly?
> after adding to my .hosts file
Check your webserver log, to see how the request came in...
curl has options to dump the request sent, and response received, it is called trace, which will will be saved to a file.
--trace
If you are missing host or header information - you can force those headers with the config option.
I would get the curl request working on the command line, and then try to implement in PHP.
the config option is
-K/--config
the options that are relevant in curl are here
--trace
This option overrides previous uses of -v/--verbose or --trace-ascii.
If this option is used several times, the last one will be used.
-K/--config
Specify the filename to -K/--config as '-' to make curl read the file from stdin.
Note that to be able to specify a URL in the config file, you need to specify it using the --url option, and not by simply writing the URL on its own line. So, it could look similar to this:
url = "http://curl.haxx.se/docs/"
Long option names can optionally be given in the config file without the initial double dashes.
When curl is invoked, it always (unless -q is used) checks for a default config file and uses it if found. The default config file is checked for in the following places in this order:
1) curl tries to find the "home dir": It first checks for the CURL_HOME and then the HOME environment variables. Failing that, it uses getpwuid() on UNIX-like systems (which returns the home dir
given the current user in your system). On Windows, it then checks for the APPDATA variable, or as a last resort the '%USERPROFILE%\Application Data'.
2) On windows, if there is no _curlrc file in the home dir, it checks for one in the same dir the curl executable is placed. On UNIX-like systems, it will simply try to load .curlrc from the deter-
mined home dir.
# --- Example file ---
# this is a comment
url = "curl.haxx.se"
output = "curlhere.html"
user-agent = "superagent/1.0"
# and fetch another URL too
url = "curl.haxx.se/docs/manpage.html"
-O
referer = "http://nowhereatall.com/"
# --- End of example file ---
This option can be used multiple times to load multiple config files.
Solution 6 - Curl
Making a request to
C:\wnmp\curl>curl.exe --trace-ascii -H 'project1.loc' -d "uuid=d99a49d846d5ae570
667a00825373a7b5ae8e8e2" http://project1.loc/Users/getSettings.xml
Resulted in the -H
log file containing:
== Info: Could not resolve host: 'project1.loc'; Host not found
== Info: Closing connection #0
== Info: About to connect() to project1.loc port 80 (#0)
== Info: Trying 127.0.0.1... == Info: connected
== Info: Connected to project1.loc (127.0.0.1) port 80 (#0)
=> Send header, 230 bytes (0xe6)
0000: POST /Users/getSettings.xml HTTP/1.1
0026: User-Agent: curl/7.19.5 (i586-pc-mingw32msvc) libcurl/7.19.5 Ope
0066: nSSL/1.0.0a zlib/1.2.3
007e: Host: project1.loc
0092: Accept: */*
009f: Content-Length: 45
00b3: Content-Type: application/x-www-form-urlencoded
00e4:
=> Send data, 45 bytes (0x2d)
0000: uuid=d99a49d846d5ae570667a00825373a7b5ae8e8e2
<= Recv header, 24 bytes (0x18)
0000: HTTP/1.1 403 Forbidden
<= Recv header, 22 bytes (0x16)
0000: Server: nginx/0.7.66
<= Recv header, 37 bytes (0x25)
0000: Date: Wed, 11 Aug 2010 15:37:06 GMT
<= Recv header, 25 bytes (0x19)
0000: Content-Type: text/html
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 24 bytes (0x18)
0000: Connection: keep-alive
<= Recv header, 25 bytes (0x19)
0000: X-Powered-By: PHP/5.3.2
<= Recv header, 56 bytes (0x38)
0000: Set-Cookie: SESSION=m9j6caghb223uubiddolec2005; path=/
<= Recv header, 57 bytes (0x39)
0000: P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 118 bytes (0x76)
0000: 6b
0004: <html><head><title>HTTP/1.1 403 Forbidden</title></head><body><h
0044: 1>HTTP/1.1 403 Forbidden</h1></body></html>
0071: 0
0074:
== Info: Connection #0 to host project1.loc left intact
== Info: Closing connection #0
My hosts file looks like:
# Copyright (c) 1993-1999 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
127.0.0.1 localhost
...
...
127.0.0.1 project1.loc
Solution 7 - Curl
For setting up virtual hosts on Apache http-servers that are not yet connected via DNS, I like to use:
curl -s --connect-to ::host-name: http://project1.loc/post.json
Where host-name ist the IP address or the DNS name of the machine on which the web-server is running. This also works well for https-Sites.