I'm pleased to announce a beta release of ZODB 3.1.2 and ZEO 2.0.2. The are three major bug fixes in this release: - A new pack() implementation for FileStorage that fixes recently discovered data corruption problems. - Atomic processing of invalidations to guarantee data consistency. - Fix for Missing.Value to prevent segfaults during comparison. This release corresponds exactly to Zope 2.6.2 beta 1, which will be released in the near future. The release is available in the usual place http://www.zope.org/Products/ZODB3.2 It has been tested with Python 2.1.3, 2.2.2, and 2.3b1 on Windows and Linux. Binary installers are available for Windows for 2.1.3 and 2.2.2. I am particularly eager to get feedback on the new pack implementation. It is a substantial rewrite of the garbage collector. It fixes several reported bugs, but I'd like to see substantial user testing to make sure we didn't add too many news ones. I've included the complete news file below. Jeremy What's new in ZODB3 3.1.2 beta 1? ================================= ZODB ---- Invalidations are now processed atomically. Each transaction will see all the changes caused by an earlier transaction or none of them. Before this patch, it was possible for a transaction to see invalid data because it saw only a subset of the invalidations. This is the most likely cause of reported BTrees corruption, where keys were stored in the wrong bucket. When a BTree bucket splits, the bucket and the bucket's parent are both modified. If a transaction sees the invalidation for the bucket but not the parent, the BTree in memory will be internally inconsistent and keys can be put in the wrong bucket. The atomic invalidation fix prevents this problem. A number of minor reference count bugs in the object cache were fixed. That's the cPickleCache.c file. It was possible for a transaction that failed in tpc_finish() to lose the traceback that caused the failure. The transaction code was fixed to report the original error as well as any errors that occur while trying to recover from the original error. ZEO --- A ZEO client will not read from its cache during cache verification. This fix was necessary to prevent the client from reading inconsistent data. The isReadOnly() method of a ZEO client was fixed to return the false when the client is connected to a read-only fallback server. The sync() method of ClientStorage and the pending() method of a zrpc connection now do both input and output. The short_repr() function used to generate log messages was fixed so that it does not blow up creating a repr of very long tuples. Storages -------- FileStorage has a new pack() implementation that fixes several reported problems that could lead to data loss. Two small bugs were fixed in DemoStorage. undoLog() did not handle its arguments correctly and pack() could accidentally delete objects created in versions. Fixed trivial bug in fsrecover that prevented it from working at all. FileStorage will use fsync() on Windows starting with Python 2.2.3. FileStorage's commit version was fixed. It used to stop after the first object, leaving all the other objects in the version. BTrees ------ Trying to store an object of a non-integer type into an IIBTree or OIBTree could leave the bucket in a variety of insane states. For example, trying b[obj] = "I'm a string, not an integer" where b is an OIBTree. This manifested as a refcount leak in the test suite, but could have been much worse (most likely in real life is that a seemingly arbitrary existing key would "go missing"). When deleting the first child of a BTree node with more than one child, a reference to the second child leaked. This could cause the entire bucket chain to leak (not be collected as garbage despite not being referenced anymore). Other minor BTree leak scenarios were also fixed. Other ----- Comparing a Missing.Value object to a C type that provide its own comparison operation could lead to a segfault when the Missing.Value was on the right-hand side of the comparison operator. The Missing class was fixed so that its coercion and comparison operations are safe. Tools ----- Four tools are now installed by setup.py: fsdump.py, fstest.py, repozo.py, and zeopack.py.