Urllib.request-urlopen opens a URL, and the server will only receive a simple request for page access, but the server does not know the browser, operating system, hardware platform and other information used to send the request, and the request without this information is often non-normal access, such as crawler.
In order to prevent this abnormal access, some websites will verify the UserAgent in the request information (its information includes hardware platform, system software, application software, and user preferences). If the UserAgent is abnormal or does not exist, then the request will be rejected (as shown in the error message above).
Add the browser camouflage [see link for method].
headers = {'User-Agent':'Mozilla/5.0 3578.98 Safari/537.36'}
url = Request(url,headers=headers)
content = urlopen(url,timeout=15).read()
Read More:
- How to Fix “HTTP error 403: forbidden” in Python 3. X
- Python: How to Obtaining Publick IP Quickly
- python3 ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833) Error
- Python error: urllib.error.HTTPError : http Error 404: not found
- When sending HTTP request, python encountered: error 54, ‘connection reset by peer’ solution
- Python 3 urllib has no URLEncode attribute
- Python + Requests +Excel+Jenkins interface automation
- Django WSGI protocol application, based on wsgiref module DIY a web framework
- The automatic token of Python interface is passed into the header
- [Solved] Selenium.common.exceptions.WebDriverException: Message: newSession
- Python: How to parses HTML, extracts data, and generates word documents
- Extracting Data from XML (Using Python to Access Web Data)
- Full explanation of SYS module of Python
- Python USES the PO design pattern for automated testing
- Linux Fatal Python error: Py_Initialize Unable to get the locale encoding
- Extracting TF-IDF keywords from text using Jieba
- Docker run xxx,E Time Elapsed: 0:00:00.000180
- About the solution of ERROR: Failed building wheel for dlib
- Python: How to Find the square root and square of numbers (Several methods)
- linux ubuntu pip search Fault: <Fault -32500: “RuntimeError: PyPI‘s XMLRPC API is currently disab