[ZODB-Dev] ZODB 3.2b3 released

[Jeremy Hylton]

We released ZODB 3.2b2 on Tuesday evening.  You can find it at the usual
place: http://www.zope.org/Products/ZODB3.2

This release contains the last new work we plan to do before the ZODB
3.2 final release.  We have fixed several bugs, including a deadlock
problem that had been reported against 3.2b2.

ZODB 3.2 has a number of new features and improvements over ZODB 3.1:
  - improve performance and stability of ZEO
  - new ZEO authentication protocol
  - new configuration language, ZConfig, for databases, storages,
    and ZEO servers
  - many bug fixes

Please give this new release a try.  We hope to have a final release
soon, and bugs reported soon are more likely to get fixed before
the final release.

I've included the NEWS.txt summary of the most recent changes.

Jeremy

What's new in ZODB3 3.2 beta 3
==============================
Release date: 23-Sep-2003

Note: The changes listed for this release include changes also made in
ZODB 3.1.x releases and ported to the 3.2 release.

This version of ZODB 3.2 is not compatible with Python 2.1.  Early
versions were explicitly designed to be compatible with Zope 2.6.
That plan has been dropped, because Zope 2.7 is already in beta
release. 

Several of the classes in ZEO and ZODB now inherit from object, making
them new-style classes.  The primary motivation for the change was to
make it easier to debug memory leaks.  We don't expect any behavior to
change as a result.

A new feature to allow removal of connection pools for versions was
ported from Zope 2.6.  This feature is needed by Zope to avoid denial
of service attacks that allow a client to create an arbitrary number
of version pools.

Fixed several critical ZEO bugs.

- If several client transactions were blocked waiting for the storage
  and one of the blocked clients disconnected, the server would
  attempt to restart one of the other waiting clients.  Since the
  disconnected client did not have the storage lock, this could lead
  to deadlock.  It could also cause the assertion "self._client is
  None" to fail.

- If a storage server fails or times out between the vote and the
  finish, the ZEO cache could get populated with objects that didn't
  make it to the storage server.

- If a client loses its connection to the server near the end of a
  transaction, it is now guaranteed to get a ClientDisconnected error
  even if it reconnects before the transaction finishes.  This is
  necessary because the server will always abort the transaction.
  In some cases, the client would never see an error for the aborted
  transaction.

- In tpc_finish(), reordered the calls so that the server's tpc_finish()
  is called (and must succeed) before we update the ZEO client cache.

- The storage name is now prepended to the sort key, to ensure a
  unique global sort order if storages are named uniquely.  This
  can prevent deadlock in some unusual cases.

Fixed several serious flaws in the implementation of the ZEO
authentication protocol.

- The smac layer would accept a message without a MAC even after the
  session key was established.

- The client never initialized its session key, so it never checked
  incoming messages or created MACs for outgoing messags.

- The smac layer used a single HMAC instance for sending and receiving
  messages.  This approach could only work if client and server were
  guaranteed to process all messages in the same total order, which
  could only happen in simple scenarios like unit tests.

Fixed a bug in ExtensionClass when comparing ExtensionClass instances.
The code could raise RuntimeWarning under Python 2.3, and produce
incorrect results on 64-bit platforms.

Fixed bug in BDBStorage that cause lead to DBRunRecoveryErrors when a
transaction was aborted after performing operations like commit
version or undo that create new references to existing pickles.

Fixed a bug in Connection.py that caused it to fail with an
AttributeError if close() was called after the database was closed.

The test suite leaves fewer log files behind, although it still leaves
a lot of junk.  The test.py script puts each tests temp files in a
separate directory, so it is easier to see which tests are causing
problems.  Unfortunately, it is still to tedious to figure out why the
identified tests are leaving files behind.

This release contains the latest and greatest version of the
BDBStorage.  This storage has still not seen testing in a production
environment, but it represents the current best design and most recent
code culled from various branches where development has occurred.

The Tools directory contains a number of small improvements, a few new
tools, and README.txt that catalogs the tools.  Many of the tools are
installed by setup.py; those scripts will now have a #! line set
automatically on Unix.

Fixed bugs in Tools/repozo.py, including a timing-dependent one that
could cause the following invocation of repozo to do a full backup when
an incremental backup would have sufficed.

A pair of new scripts from Jim Fulton can be used to synthesize
workloads and measure ZEO performance:  see zodbload.py and
zeoserverlog.py in the Tools directory.  Note that these require
Zope.

Tools/checkbtrees.py was strengthened in two ways:

- In addition to running the _check() method on each BTree B found,
  BTrees.check.check(B) is also run.  The check() function was written
  after checkbtrees.py, and identifies kinds of damage B._check()
  cannot find.

- Cycles in the object graph no longer lead to unbounded output.
  Note that preventing this requires remembering the oid of each
  persistent object found, which increases the memory needed by the
  script.