Commit 3580ddd8 authored by Jason Madden's avatar Jason Madden Committed by Jim Fulton

Speed up ZODB.blob.BushyLayout.oid_to_path on Python 3 (#161)

Profiling (https://github.com/zodb/zodbshootout/pull/32/) showed that
this method was the only blob-related method that showed up in a test
of creating blobs, other than those that actually performed IO.

With this change its total and cumulative time drops from 0.003/0.004
to 0.001/0.002 in a small benchmark. Blobs created per second shows a
small but consistent improvement.

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
      200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
      101    0.003    0.000    0.003    0.000 {built-in method mkdir}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
        1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     5633    0.001    0.000    0.002    0.000 {built-in method isinstance}

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      101    0.005    0.000    0.005    0.000 {built-in method mkdir}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
        1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
   ....
      200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)
parent 4b500e78
......@@ -5,7 +5,7 @@
5.2.4 (unreleased)
==================
- Nothing changed yet.
- Optimize getting the path to a blob file.
5.2.3 (2017-04-11)
......
......@@ -574,18 +574,21 @@ class BushyLayout(object):
r'(0x[0-9a-f]{1,2}\%s){7,7}0x[0-9a-f]{1,2}$' % os.path.sep)
def oid_to_path(self, oid):
directories = []
# Create the bushy directory structure with the least significant byte
# first
for byte in ascii_bytes(oid):
if isinstance(byte,INT_TYPES): # Py3k iterates byte strings as ints
hex_segment_bytes = b'0x' + binascii.hexlify(bytes([byte]))
hex_segment_string = hex_segment_bytes.decode('ascii')
else:
hex_segment_string = '0x%s' % binascii.hexlify(byte)
directories.append(hex_segment_string)
oid_bytes = ascii_bytes(oid)
hex_bytes = binascii.hexlify(oid_bytes)
assert len(hex_bytes) == 16
directories = [b'0x' + hex_bytes[x:x+2]
for x in range(0, 16, 2)]
return os.path.sep.join(directories)
if bytes is not str: # py3
sep_bytes = os.path.sep.encode('ascii')
path_bytes = sep_bytes.join(directories)
return path_bytes.decode('ascii')
else:
return os.path.sep.join(directories)
def path_to_oid(self, path):
if self.blob_path_pattern.match(path) is None:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment