These days when I do crawler with Python, I always encounter Python URLError: < urlopen error timed out> The more threads open, the more frequently they occur. Later proved that the program did not have a problem, checked the information on the Internet, the network is blocked, more than a few times to try it.
For example, the original code looks like this:
For example, the original code looks like this:
if headers:
req=urllib2.Request(url,headers=headers)
else :
req=urllib2.Request(url)
opener=urllib2.build_opener(cookieproc)
urllib2.install_opener(opener)
page=urllib2.urlopen(req,timeout=5).read()
can be changed after adding a loop:
if headers:
req=urllib2.Request(url,headers=headers)
else :
req=urllib2.Request(url)
opener=urllib2.build_opener(cookieproc)
urllib2.install_opener(opener)
global Max_Num
Max_Num=6
for i in range(Max_Num):
try:
page=urllib2.urlopen(req,timeout=5).read()
break
except:
if i < Max_Num-1:
continue
else :
print 'URLError: <urlopen error timed out> All times is failed '
This exception can generally be resolved
Read More:
- python:urllib2.URLError urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
- URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
- urllib.error.URLError: <urlopen error [WinError 10060] problem solving
- seaborn.load_ Data set error urlerror: < urlopen error [winerror 10060] the connection attempt failed because the connecting party did not reply correctly after a period of time or the connected host did not respond
- A download error occurred while downloading data from pytorch. Urllib.error.urlerror: < urlopen error [SSL: certificate_verify_failed]
- Resourceaccessexception: I / O error on post request for and connection timed out
- Synergy Ubuntu 18 + win7 timed out or new client is unresponsive
- java.sql.SQLException : IO exception: socket read timed out
- Putty:Network error:Connection timed out
- When idea generates a spring boot project, error: read timed out appears
- NBU recovery report error socket read failed: errno = 119-System call timed out
- Sync with Gradle for project ‘XXXX‘ failed: Connection timed out: connect
- Ubuntu 20.04 GPG: KeyServer receive failed: connection timed out when installing ROS configuration key
- esp32:A fatal error occurred: Timed out waiting for packet header
- java.sql.SQLException : IO error: socket read timed out!
- [resolved] exception java.net.ConnectException : Error opening socket to server Connection timed out.
- Error: JMeter monitors Linux system performance java.net.ConnectException : Connection timed out: connect
- RTMP_Connect0, failed to connect socket. 110 (Connection timed out)
- Debian/Ubuntu uses proxy for gpg and apt-key-error resolution: gpg: keyserver receive failed: Connection timed out
- pycharm Failed to connect to github.com port 443: Timed out