Commit 898b2e87 authored by Levin Zimmermann's avatar Levin Zimmermann

fixup! fixup! fixup! ZBigFile: Add ZBlk format option 'h' (heuristic) (4)

'ZBlk0' and 'ZBlk1' always only change the block that's explicitly altered,
while zblk format 'auto' may also alter blocks different to the explicitly
changed blocks. In other words, as @kirr noted in nexedi/wendelin.core!20 (comment 198863),
format 'auto' break the invariant

    "transaction changes ZBlk objects only for modified blocks of array" [1]

Breaking this invariant leads to two failed WCFS tests -
'test_wcfs_crash_old_data' and 'test_wcfs_watch_vs_access'.

As suggested in nexedi/wendelin.core!20 (comment 198863)
instead of patching tests, we rather try not to break this invariant
with format 'auto'. This is exactly what this patch does. Instead of
changing the previously filled up block, we immediately change a blocks
format if we can assume that it is already full.

[1] nexedi/wendelin.core!20 (comment 198863)
parent 01364302
......@@ -599,13 +599,15 @@ class ZBigFile(LivePersistent):
append = (new_data[:len(old_data)] == old_data)
small = (ndelta < 0.5*self.blksize)
# append - migrate previously filled-up block to ZBlk0 for fast reads
# - for current block use ZBlk1 if the append is small and ZBlk0 otherwise
# append - if block is already full, use ZBlk0 for fast reads
# - otherwise use ZBlk1 if the append is small and ZBlk0 otherwise
if append:
if not zblk and blk > 0: # is new zblk?
zblk_prev = self.blktab.get(blk-1)
if zblk_prev is not None and type(zblk_prev) is not ZBlk0:
self._setzblk(blk-1, zblk_prev, zblk_prev.loadblkdata(), ZBlk0)
# NOTE: There can always be (stripped) 0 at the end of a block, so that
# a full block isn't necesarrily as big as blksize. We therefore use an
# approximate definition of fullness here: if a block is mostly filled
# with != 0 data, we assume it's full.
if len(new_data) >= (self.blksize * 0.99):
return ZBlk0
return ZBlk1 if small else ZBlk0
# all other changes - use ZBlk1 if the change is small and ZBlk0 otherwise
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment