is thrown because of an error in the title:

Message: Tried to run command without establishing a connection

interprets:

to start with my crawler architecture, I use firefox+selenium. The error above is caused by the fact that after the browser exits, the crawler needs the browser to execute some commands, so the error appears. The second problem comes:

Why does the

browser crash automatically?Generally speaking, it is because it is running out of resources. What resources?Memory resources, the browser is very occupied memory, and some crawlers in order to speed up the crawler will let the browser do cache,

this results in the browser taking up more memory

solution:

1. According to the resource occupied by the crawler, increase the memory

appropriately

2. Slow down the crawler speed and make the browser a little idle, especially in the crawler that starts multiple browsers

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — the 2019-04-24 update — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

the above statement is not wrong, but some is not taken into account, the browser will actually the main reason for the collapse because the browser memory leaks, that is to say, because the default browser is open caching function,

as the crawler runs, the cache of the browser will become larger and larger, which will eventually lead to the memory leak of the browser ( premise is that the browser does not restart the crawler, if the browser restarts after a period of time, there will be no problem ),

as for how to disable browser cache in crawlers, this is mentioned in my other blog, but not

reproduced in: https://www.cnblogs.com/gunduzi/p/10600152.html