1. Problem description
In a web project built with gunicorn + flash, because the machine learning model is used, the model needs to be loaded when requesting the interface for the first time. The model is loaded quickly when running locally, so it runs normally. The application is deployed to the server using docker (the model is mounted to the container through volume), The first request to load the model takes a long time, and the following errors are reported:
[2021-09-11 07:22:33 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:12)
[2021-09-11 07:22:35 +0000] [12] [INFO] Worker exiting (pid: 12)
Gunicorn ‘green Unicorn’ is a python WSGI HTTP server for UNIX. It is a pre forked worker model transplanted from Ruby’s Unicorn project. Gunicorn server is widely compatible with various web frameworks, with simple implementation, less use of server resources and quite fast speed.
2. Cause analysis
It can be seen from the error message that gunicorn’s worker process timed out, causing the process to exit and restart
refer to the official document of gunicorn and the description shown in the figure below:
3. Solution
From the documentation on the official website, we can know that the default timeout of gunicorn is 30s. If it exceeds 30s, the worker process will be killed and restarted.
Therefore, to solve this problem, you only need to set a large timeout: gunicorn to set the timeout
gunicorn -w 2 -b 0.0.0.0:4343 manage:app --timeout 120
Add: -- timeout 120 to gunicorn’s startup command, indicating that the timeout is set to 120 seconds. Configure the timeout in the gunicorn. Conf.py configuration file of gunicorn
workers = 4 # Define the number of processes to be opened for processing requests at the same time, adjusted appropriately according to site traffic
worker_class = "gevent" # Use the gevent library to support asynchronous processing of requests and improve throughput
bind = "0.0.0.0:8500" # Listen to IP relaxation to facilitate communication between Dockers and between Dockers and hosts
timeout = 120 # Set the timeout to 120 seconds
For more configuration of the
gunicorn. Conf.pyfile, see the official website
Read More:
- Crawler overtime error socket.timeout: timed out/NameError: name ‘socket‘ is not defined
- Gunicorn Flask Error: [ERROR] Socket error processing request
- [Solved] Yolov5 Deep Learning Error: RuntimeError: DataLoader worker (pid(s) 2516, 1768) exited unexpectedly
- [Solved] ERROR PythonRunner: Python worker exited unexpectedly (crashed)
- [Solved] PyTorch Caught RuntimeError in DataLoader worker process 0和invalid argument 0: Sizes of tensors mus
- [Solved] Jupyter Notebook Error: SparkException: Python worker failed to connect back
- [Solved] ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memor
- Solution for Python3.7 import gevent module error
- [Solved] Model training Error: _pickle.PicklingError: Can’t pickle
- Decode PUP data Error when reading radar data [How to Solve]
- Python + Requests +Excel+Jenkins interface automation
- [Solved] Vite build & Flask Error: Failed to load module script. Strict MIME type checking is enforced
- How to Solve Pytorch eval Stuck Error
- ModuleNotFoundError: No module named ‘requests‘ [How to Solve]
- Failed to establish a new connection: [winerror 10048] in the requests thread pool, the interface is called circularly to get the datagram error
- Pytorch Run Error: BrokenPipeError [How to Solve]
- [Solved] Flask Application Context Error: RuntimeError: Working outside of application context.
- Python Requests Error: Max retries exceeded with url
- Python Error: Failed to establish a new connection: [Errno -2] Name or service not known
- [Solved] AttributeError: Manager isn‘t available; ‘auth.User‘ has been swapped for ‘