[Zodb-checkins] CVS: ZODB/src/ZODB/FileStorage - fspack.py:1.11

[Tim Peters]

Update of /cvs-repository/ZODB/src/ZODB/FileStorage
In directory cvs.zope.org:/tmp/cvs-serv29374/src/ZODB/FileStorage

Modified Files:
	fspack.py 
Log Message:
FileStoragePacker:  open the filestorage in unbuffered mode.  Transactions
can still be in progress, and they're written to the same physical file via
a different file object.  Using buffered I/O in the packer creates the
possiblity for the packer to see stale data from its file object's stdio
buffers.  This is now known to happen under Linux, Gentoo, OS X, Cygwin,
and Debian.  It apparently doesn't happen under native Windows, which is
why everyone except me has been seeing the new checkPackLotsWhileWriting
test fail.

This will probably need to be backported everywhere, but first I want to
see in which new way checkPackLotsWhileWriting fails on 48 Linux boxes
overnight <wink>.


=== ZODB/src/ZODB/FileStorage/fspack.py 1.10 => 1.11 ===
--- ZODB/src/ZODB/FileStorage/fspack.py:1.10	Wed Mar 17 15:50:06 2004
+++ ZODB/src/ZODB/FileStorage/fspack.py	Thu Mar 18 18:41:47 2004
@@ -416,7 +416,14 @@
     # progress after it).
     def __init__(self, path, stop, la, lr, cla, clr, current_size):
         self._name = path
-        self._file = open(path, "rb")
+        # Caution:  It's critical that the file be opened in unbuffered mode.
+        # The code used to leave off the trailing 0 argument, and then on
+        # every platform except native Windows it was observed that we could
+        # read stale data from the tail end of the file -- keep in mind that
+        # transactions can still be in progress throughout much of packing,
+        # and are written to the same physical file but via a distinct Python
+        # file object.
+        self._file = open(path, "rb", 0)
         self._stop = stop
         self.locked = 0
         self.file_end = current_size