Difficulties with MVCC implementation was: [ZODB-Dev] Potential BTrees splitting bug

[Tim Peters]

[Dieter Maurer]
> I found the likely cause for the test failure: the limited
> time resolution.
>
>   The idea behind my MVCC implementation is to deliver the state
>   as it has been at transaction start.
>   Due to the limited time resolution, the transaction start
>   cannot be determined precisely but only within the resolution limit.

I'm assuming this became "a problem" for you on one particular system
because that system was exceptionally fast.  If so, you could probably
provoke problems much more easily on a Windows box, where time.time()
updates only 18.2 times per second (this is in some sense the opposite of
what happens on Linux:  on Windows time.time() has only ~= 0.055 second
resolution while on Linux time.time() typically has microsecond resolution;
but time.clock() on Windows typically has sub-microsecond resolution while
time.clock on Linux typically has 0.01 second resolution).

> As a consequence, we cannot trust a state defined near the transaction
> start. I will have to raise a "ReadConflictError" in this case.

Brrr.  That is disappointing.

> Hopefully, the ZODB ensures that different transaction commits get
> different serials (as this is necessary to detect the need for
> conflict resolution).

Right; if you're curious, the bottom-level relevant code is in TimeStamp.c's
TimeStamp_laterThan() (and, yes, that could sure use some comments!).

> In this case, a more elegant solution might be possible, synchronizing
> transaction start time with last transaction serials picky-packed in
> invalidation messages.  However, this would require a protocol change
> (beside other problems such as coordinating multiple connections).

We (Zope Corp) can't add MVCC to ZODB 3.1 because that's in maintenance mode
now, and new features don't get added after feature freeze.  The same
applies to ZODB 3.2 (not yet released, but entered feature freeze when the
first beta got released).  The protocol in 3.2 already changed, though, and
in 3.2 invalidation messages include the transaction id.  When Zope releases
an MVCC implementation, it will have to be in 3.3 (maybe not under that
name -- I don't know yet know what it will be called).