is now ubiquitous for sites that begin with HTTPS, and urllib2 can request SSL certificates for HTTPS, just like a web browser, if the site’s SSL certificate is CA certified, it can be accessed normally,

if SSL certificate authentication fails, or if the operating system does not trust the server’s security certificate, such as when the browser visits the 12306 site, it warns the user that the certificate is not trusted. (it is said that 12306 website certificate is done by oneself, did not pass CA authentication)

warns the user that the certificate is not trusted

urllib2 will report SSLError when accessing:

import urllib2
url = "https://www.12306.cn/mormhweb/"
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"}
request = urllib2.Request(url, headers = headers)
response = urllib2.urlopen(request)
print response.read()

operation results:

urllib2.URLError: < urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

therefore, if we encounter such a site in the future, we need to process SSL certificates separately, let the program ignore SSL certificate validation errors, can be normal access.

import urllib
import urllib2

# 1. 导入Python SSL处理模块
import ssl
 
# 2. 表示忽略未经核实的SSL证书认证
context = ssl._create_unverified_context()
 
url = "https://www.12306.cn/mormhweb/"
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"}
request = urllib2.Request(url, headers = headers)
 
# 3. 在urlopen()方法里 指明添加 context 参数
response = urllib2.urlopen(request, context = context)
print response.read()

Summary: the solution includes the following two ways:

1. Use SSL to create an unauthenticated context, passing in the context parameter

in urlopen

import ssl
import urllib2
context = ssl._create_unverified_context()
print urllib2.urlopen("https://www.12306.cn/mormhweb/", context=context).read()

2. Global cancel certificate validation

import ssl
import urllib2
 
ssl._create_default_https_context = ssl._create_unverified_context
print urllib2.urlopen("https://www.12306.cn/mormhweb/").read()

About CA

CA(Certificate Authority) is the short name of digital Certificate certification Authority, which refers to a trusted third-party organization that issues, manages and abolishes digital certificates,

The

CA checks the legitimacy of the certificate holder’s identity and issues certificates to prevent certificates from being forged or tampered with, as well as to manage certificates and keys.

id can be used to prove identity in real life, so in the network world, digital certificate is id. Different from the real life, not every Internet user has a digital certificate, often only when a person needs to prove their identity to use the digital certificate.

the average user generally does not need, because the website does not care who visits the website, now the website only CARES about the traffic. But in turn, the site needs to prove its identity.