Commit 898b2e87 authored by Levin Zimmermann's avatar Levin Zimmermann

fixup! fixup! fixup! ZBigFile: Add ZBlk format option 'h' (heuristic) (4)

'ZBlk0' and 'ZBlk1' always only change the block that's explicitly altered,
while zblk format 'auto' may also alter blocks different to the explicitly
changed blocks. In other words, as @kirr noted in nexedi/wendelin.core!20 (comment 198863),
format 'auto' break the invariant

    "transaction changes ZBlk objects only for modified blocks of array" [1]

Breaking this invariant leads to two failed WCFS tests -
'test_wcfs_crash_old_data' and 'test_wcfs_watch_vs_access'.

As suggested in nexedi/wendelin.core!20 (comment 198863)
instead of patching tests, we rather try not to break this invariant
with format 'auto'. This is exactly what this patch does. Instead of
changing the previously filled up block, we immediately change a blocks
format if we can assume that it is already full.

[1] nexedi/wendelin.core!20 (comment 198863)
parent 01364302
...@@ -599,13 +599,15 @@ class ZBigFile(LivePersistent): ...@@ -599,13 +599,15 @@ class ZBigFile(LivePersistent):
append = (new_data[:len(old_data)] == old_data) append = (new_data[:len(old_data)] == old_data)
small = (ndelta < 0.5*self.blksize) small = (ndelta < 0.5*self.blksize)
# append - migrate previously filled-up block to ZBlk0 for fast reads # append - if block is already full, use ZBlk0 for fast reads
# - for current block use ZBlk1 if the append is small and ZBlk0 otherwise # - otherwise use ZBlk1 if the append is small and ZBlk0 otherwise
if append: if append:
if not zblk and blk > 0: # is new zblk? # NOTE: There can always be (stripped) 0 at the end of a block, so that
zblk_prev = self.blktab.get(blk-1) # a full block isn't necesarrily as big as blksize. We therefore use an
if zblk_prev is not None and type(zblk_prev) is not ZBlk0: # approximate definition of fullness here: if a block is mostly filled
self._setzblk(blk-1, zblk_prev, zblk_prev.loadblkdata(), ZBlk0) # with != 0 data, we assume it's full.
if len(new_data) >= (self.blksize * 0.99):
return ZBlk0
return ZBlk1 if small else ZBlk0 return ZBlk1 if small else ZBlk0
# all other changes - use ZBlk1 if the change is small and ZBlk0 otherwise # all other changes - use ZBlk1 if the change is small and ZBlk0 otherwise
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment