Use subprocess to execute the command line, and the pipeline is blocked

When using subprocess to execute a series of CMD commands in Python, occasionally there will be blocking, and the command does not continue to execute.

reason:

The pipe of a subprocess has a size. Before python2.6.11, the size of pipe was the size of file page (4096 on i386),
# after 2.6.11, it became 65536. Therefore, when the output content exceeded 65536, it would cause blocking

solve:

1. Use tempfile to expand the cache;

2. Remove unnecessary output to reduce the output

In scheme 1, the temporary file tempfile is used to expand the cache

out_ temp = tempfile.SpooledTemporaryFile (bufsize=10 * 1000)
fileno = out_ temp.fileno ()
process = subprocess.Popen (cmd, stdout=fileno, stderr=fileno, shell=True) # stdout= subprocess.PIPE ,

Scheme 2, according to the actual situation to reduce unnecessary data.


Read More: