requests: how to disable / bypass proxy

PythonPython Requests

Python Problem Overview


I am getting an url with:

r = requests.get("http://myserver.com")

As I can see in the 'access.log' of "myserver.com", the client's system proxy is used. But I want to disable using proxies at all with requests.

Python Solutions


Solution 1 - Python

The only way I'm currently aware of for disabling proxies entirely is the following:

  • Create a session
  • Set session.trust_env to False
  • Create your request using that session

import requests

session = requests.Session()
session.trust_env = False

response = session.get('http://www.stackoverflow.com')

This is based on this comment by Lukasa and the (limited) documentation for requests.Session.trust_env.

Note: Setting trust_env to False also ignores the following:

  • Authentication information from .netrc (code)
  • CA bundles defined in REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE (code)

If however you only want to disable proxies for a particular domain (like localhost), you can use the NO_PROXY environment variable:

import os
import requests

os.environ['NO_PROXY'] = 'stackoverflow.com'

response = requests.get('http://www.stackoverflow.com')

Solution 2 - Python

You can choose proxies for each request. From the docs:

import requests

proxies = {
  "http": "http://10.10.1.10:3128",
  "https": "http://10.10.1.10:1080",
}

requests.get("http://example.org", proxies=proxies)

So to disable the proxy, just set each one to None:

import requests

proxies = {
  "http": None,
  "https": None,
}

requests.get("http://example.org", proxies=proxies)

Solution 3 - Python

The way to stop requests/urllib from proxying any requests is to set the the no_proxy (or NO_PROXY) environment variable to * e.g. in bash:

export no_proxy='*'

Or from Python:

import os
os.environ['no_proxy'] = '*' 

To understand why this works is because the urllib.request.getproxies function first checks for any proxies set in the environment variables (e.g. http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY, etc) or if none are set then it will check for system configured proxies using platform specific calls (e.g. On MacOS it will check using the system scutil/configd interfaces, and on Windows it will check the Registry). As mentioned in the comments if any proxy variables are set you can reset them as @udani suggested, or unset them like this from Python:

del os.environ['HTTP_PROXY']

Then when urllib attempts to use any proxies the proxyHandler function it will check for the presence and setting of the no_proxy environment variable - which can either be set to specific hostnames as mentioned above or it can be set the special * value whereby all hosts bypass the proxy.

Solution 4 - Python

requests library respects environment variables. http://docs.python-requests.org/en/latest/user/advanced/#proxies

So try deleting environment variables HTTP_PROXY and HTTPS_PROXY.

import os
for k in list(os.environ.keys()):
    if k.lower().endswith('_proxy'):
        del os.environ[k]

Solution 5 - Python

With Python3, jtpereyda's solution didn't work, but the following did:

proxies = {
    "http": "",
    "https": "",
}

Solution 6 - Python

 r = requests.post('https://localhost:44336/api/',data='',verify=False)

I faced the same issue when connecting with localhost to access my .net backend from a Python script with the request module.

I set verify to False, which cancels the default SSL verification.

P.s - above code will throw a warning that can be neglected by below one

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
r=requests.post('https://localhost:44336/api/',data='',verify=False)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questiont777View Question on Stackoverflow
Solution 1 - PythonLukas GrafView Answer on Stackoverflow
Solution 2 - PythonjtpereydaView Answer on Stackoverflow
Solution 3 - PythonPierzView Answer on Stackoverflow
Solution 4 - PythonKostasTView Answer on Stackoverflow
Solution 5 - PythonOriol VilasecaView Answer on Stackoverflow
Solution 6 - Pythonritwik chakrabortyView Answer on Stackoverflow