ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

PythonPython 3.xSsl

Python Problem Overview


I am trying to use the betbrain.py from Github (https://github.com/gto76/betbrain-scraper) that has the following code:

#!/usr/bin/python3
#
# Usage: betbrain.py [URL or FILE] [OUTPUT-FILE]
# Scrapes odds from passed betbrain page and writes them to
# stdout, or file if specified.

import os
import sys
import urllib.request


from bs4 import BeautifulSoup
from http.cookiejar import CookieJar

import parser_betbrain
import printer

DEFAULT_URL = 'https://www.betbrain.com/football/england/premier-league/#!/matches/'


# If no arguments are present, it parses the default page.
# Argument can be an URL or a local file.
def main():
  html = getHtml(sys.argv)
  soup = BeautifulSoup(html, "html.parser")
  matches = parser_betbrain.getMatches(soup)
  string = printer.matchesToString(matches)
  output(string, sys.argv)

def getHtml(argv):
  if len(argv) <= 1:
    return scrape(DEFAULT_URL)
  elif argv[1].startswith("http"):
    return scrape(argv[1])
  else:
    return readFile(argv[1])

# Returns html file located at URL.
def scrape(url):
  cj = CookieJar()
  opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
  try:
    return opener.open(url)
  except ValueError:
    error("Invalid URL: " + url)

def readFile(path):
  try:
    return open(path, encoding='utf8')
  except IOError:
    error("Invalid input filename: " + path)

def output(string, argv):
  if len(argv) <= 2:
    print(string)
  else:
    writeFile(argv[2], string)

def writeFile(path, string):
  try:  
    fo = open(path, "w", encoding='utf8')
    fo.write(string);
    fo.close()
  except IOError:
    error("Invalid output filename: " + path)

def error(msg):
  msg = os.path.basename(__file__)+": "+msg
  print(msg, file=sys.stderr)
  sys.exit(1)

if __name__ == '__main__':
  main()

However when run it comes back with this error

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Daniel/Desktop/betbrain-scraper-master 2/betbrain.py", line 71, in <module>
    main()
  File "/Users/Daniel/Desktop/betbrain-scraper-master 2/betbrain.py", line 22, in main
    html = getHtml(sys.argv)
  File "/Users/Daniel/Desktop/betbrain-scraper-master 2/betbrain.py", line 30, in getHtml
    return scrape(DEFAULT_URL)
  File "/Users/Daniel/Desktop/betbrain-scraper-master 2/betbrain.py", line 41, in scrape
    return opener.open(url)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

How can I fix this problem? I am running Python 3.6.0 on MacOS 10.12.1

Python Solutions


Solution 1 - Python

Open a terminal and take a look at:

/Applications/Python 3.6/Install Certificates.command

Python 3.6 on MacOS uses an embedded version of OpenSSL, which does not use the system certificate store. More details here.

(To be explicit: MacOS users can probably resolve by opening Finder and double clicking Install Certificates.command)

Solution 2 - Python

I ran this on MacOS /Applications/Python\ 3.6/Install\ Certificates.command

Solution 3 - Python

On CentOS Linux, Python3.6, I edited this file (make a backup copy first)

/usr/lib/python3.6/site-packages/certifi/cacert.pem

to the end of the file, I added my public certificate from my .pem file. you should be able to obtain the .pem file from your ssl certificate provider.

Solution 4 - Python

I have encountered the same issue in the official Python docker and solved it by running apt-get update && apt-get install ca-certificates before running any of the nltk code.

Solution 5 - Python

In windows adding these 2 lines on the top can be a quick fix:

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

Solution 6 - Python

Here's how I fixed it:

  1. Opened the Install Cerificates.Command.The shell script got executed.
  2. Opened the Python 3.6.5 and typed in nltk.download().The download graphic window opened and all the packages got installed.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionDan LittleView Question on Stackoverflow
Solution 1 - PythonScovettaView Answer on Stackoverflow
Solution 2 - PythonNagalakshmi SriramaView Answer on Stackoverflow
Solution 3 - PythonJoe JadamecView Answer on Stackoverflow
Solution 4 - PythonSimas JoneliunasView Answer on Stackoverflow
Solution 5 - PythonDev. RafiqView Answer on Stackoverflow
Solution 6 - PythonS.Harini SudarsanView Answer on Stackoverflow