Commit d0e1d819 authored by Tim Peters's avatar Tim Peters

FileStoragePacker: open the filestorage in unbuffered mode. Transactions

can still be in progress, and they're written to the same physical file via
a different file object.  Using buffered I/O in the packer creates the
possiblity for the packer to see stale data from its file object's stdio
buffers.  This is now known to happen under Linux, Gentoo, OS X, Cygwin,
and Debian.  It apparently doesn't happen under native Windows, which is
why everyone except me has been seeing the new checkPackLotsWhileWriting
test fail.

This will probably need to be backported everywhere, but first I want to
see in which new way checkPackLotsWhileWriting fails on 48 Linux boxes
overnight <wink>.
parent 07b2ddad
...@@ -416,7 +416,14 @@ class FileStoragePacker(FileStorageFormatter): ...@@ -416,7 +416,14 @@ class FileStoragePacker(FileStorageFormatter):
# progress after it). # progress after it).
def __init__(self, path, stop, la, lr, cla, clr, current_size): def __init__(self, path, stop, la, lr, cla, clr, current_size):
self._name = path self._name = path
self._file = open(path, "rb") # Caution: It's critical that the file be opened in unbuffered mode.
# The code used to leave off the trailing 0 argument, and then on
# every platform except native Windows it was observed that we could
# read stale data from the tail end of the file -- keep in mind that
# transactions can still be in progress throughout much of packing,
# and are written to the same physical file but via a distinct Python
# file object.
self._file = open(path, "rb", 0)
self._stop = stop self._stop = stop
self.locked = 0 self.locked = 0
self.file_end = current_size self.file_end = current_size
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment