Commit ce75e16e authored by Kirill Smelkov's avatar Kirill Smelkov Committed by Romain Courteaud

component/python-2.7: Lib/subprocess: Speedup close_fds=True

@romain reports that Popen(close_fds=True) is slow on py2. Let's semantically
backport from py3 how to close only actually opened file descriptors instead of
whole 3..`ulimit -n` range.

Attached test benchmark shows the following results with `ulimit -n`=65K:

Before this patch:

    $ ./bin/python2.7 ~/x.py
    close_fds=False:
    0.001251 s/call
    0.001337 s/call
    0.001486 s/call

    close_fds=True:
    0.017973 s/call
    0.018152 s/call
    0.018204 s/call

After the patch:

    $ ./bin/python2.7 ~/x.py
    close_fds=False:
    0.001391 s/call
    0.001416 s/call
    0.001570 s/call

    close_fds=True:
    0.001469 s/call
    0.001479 s/call
    0.001491 s/call

i.e. ~12x speedup.

References on this subject are in the patch itself.

The test benchmark is below:

---- 8< ----
import timeit
from subprocess import check_call

def f():
    check_call(['true'], close_fds=False)
def g():
    check_call(['true'], close_fds=True)

N=3
n=100
print 'close_fds=False:'
for i in range(N):
    print '%.6f s/call' % (timeit.timeit(f, number=n) / n)
print
print 'close_fds=True:'
for i in range(N):
    print '%.6f s/call' % (timeit.timeit(g, number=n) / n)

/helped-by @jm
parent 7cf8d439
Pipeline #19119 failed with stage
in 0 seconds