component/python-2.7: Lib/subprocess: Speedup close_fds=True
@romain reports that Popen(close_fds=True) is slow on py2. Let's semantically backport from py3 how to close only actually opened file descriptors instead of whole 3..`ulimit -n` range. Attached test benchmark shows the following results with `ulimit -n`=65K: Before this patch: $ ./bin/python2.7 ~/x.py close_fds=False: 0.001251 s/call 0.001337 s/call 0.001486 s/call close_fds=True: 0.017973 s/call 0.018152 s/call 0.018204 s/call After the patch: $ ./bin/python2.7 ~/x.py close_fds=False: 0.001391 s/call 0.001416 s/call 0.001570 s/call close_fds=True: 0.001469 s/call 0.001479 s/call 0.001491 s/call i.e. ~12x speedup. References on this subject are in the patch itself. The test benchmark is below: ---- 8< ---- import timeit from subprocess import check_call def f(): check_call(['true'], close_fds=False) def g(): check_call(['true'], close_fds=True) N=3 n=100 print 'close_fds=False:' for i in range(N): print '%.6f s/call' % (timeit.timeit(f, number=n) / n) print print 'close_fds=True:' for i in range(N): print '%.6f s/call' % (timeit.timeit(g, number=n) / n) /helped-by @jm
Status | Job ID | Name | Coverage | ||||||
---|---|---|---|---|---|---|---|---|---|
External | |||||||||
passed |
#333921
external
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:18:46
|
||||||
passed |
#333868
external
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:38:36
|
||||||
passed |
#333695
external
|
SlapOS.SlapRunner.ResilienceTest-Master |
00:36:25
|
||||||
failed |
#333879
external
|
SlapOS.SlapRunner.ResilienceTest-Master.ERP5 |
02:16:30
|
||||||
failed |
#333688
external
|
SlapOS.SoftwareReleases.IntegrationTest-Master.Python2 |
05:06:49
|
||||||
failed |
#333690
external
|
SlapOS.SoftwareReleases.IntegrationTest-Master.Python3 |
04:21:54
|
||||||
failed |
#333793
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:22:26
|
||||||
failed |
#333719
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:15:51
|
||||||
failed |
#333883
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:00:42
|
||||||
failed |
#333895
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:00:28
|
||||||
failed |
#333909
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:00:13
|
||||||
failed |
#333781
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:30
|
||||||
failed |
#333706
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:08:09
|
||||||
failed |
#333818
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:18
|
||||||
failed |
#333831
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:00:56
|
||||||
failed |
#333843
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:14
|
||||||
failed |
#333856
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:00
|
||||||
failed |
#333805
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:15
|
||||||
failed |
#333731
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:16
|
||||||
failed |
#333744
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:19
|
||||||
failed |
#333756
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:28
|
||||||
failed |
#333769
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:01:19
|
||||||