[ZODB-Dev] How can reading trigger a ConflictError?

[Jeremy Hylton]

>>>>> "GW" == gward  <gward@mems-exchange.org> writes:

  [Greg describes a scenario where he gets a ConflictError on a read.]

  GW> So why does it work like this?  I thought the whole point of
  GW> transactions was to isolate concurrent processes from each
  GW> other; shouldn't process B happily see the "old" picture of the
  GW> database until it does something to resync (eg. commit or abort
  GW> a transaction), at which point it sees the "new" picture as
  GW> committed by A?  Or is this a naive understanding of
  GW> transactions?  Or do I just not understand ZODB/ZEO's
  GW> transaction model?  (Is it documented anywhere yet?)

[Blast from the past, eh?]

Greg asked this question.  I always intended to answer, but it took me
a while to figure out what the right answer is :-).

The basic scenario is two have two different processes committing
transactions using ZEO.  When one process updates an object, the ZEO
server informs all the other clients that the object has changed.  The
clients need this information to remove the objects from the in-memory
caches used by each connection (thread).  Because each connection must
present a consistent view of the database, it can only perform those
invalidations at transaction boundaries.  

So to answer a part of your first question: You see the updated object
after your abort a transaction, because you're at a boundary where the
connection can update your cache.

ZODB will raise a ConflictError on a read if it sees that there are
invalidation messages that have arrived after the transaction starts.
It raises this exception, because the transaction *might* see an
inconsistent view of the database.  Say an invalidation arrives for
transaction T1, which updated O1, O2, and O3.  The current
transactions has already read O1 and will read O2 in the future.

At this point you ask: Shouldn't the current transaction see the "old"
picture of the database?  It would be nice...  The current
architecture always reads the current revision of an object from the
storage.

So the current transaction attempts to read O2.  ZODB loads the object
and then sees that there is an invalidation message that for O2 that
is queued waiting for the next transaction boundary.  ZODB raises a
ConflictError during the read, because the load returned a revision of
the object that may not be consistent with other objects it has read.

In the case of T1, the current transaction read the revision of O1
commited before T1 and the revision of O2 commited by T1.  This is
inconsistent.  

The implemenation, however, is fairly conservative, because it will
raise a ConflictError whenever it sees an invalidation for an object
it has read.  This isn't always necessary.  Say the current
transaction is going to read O2 and only O2.  Then there's a chance
that it would see a consistent view of the database despite the
invalidation.  ZODB doesn't do enough bookkeeping to allow this to
work in cases where it is consistent.  (I think it would need to track
all objects read by a transaction and compare them versus
invalidations.) 

I was also bothered that ConflictError can get raised anywhere in a
persistent application, not just at places where the transaction
machinery is visible.  The use of ZODB is mostly transparent and I was
hoping that the ConflictError could only be raised at places where I
interact with the transaction manager, e.g. get_transaction().commit().
This can't be supported because the transaction manager only keeps
track of writes.  A read-only transaction would see an inconsistent
view of the database and the transaction manager would never catch
it.  It's also a bit of a performance win; if we detect a conflict on
a read, we can abort the transaction before it does more unnecessary
work. 

We'd like to change this behavior by introducing a multi-version
concurrency control.  The idea is that each transaction has a time
range associated with it and attempts to read revisions of objects
that were valid during that time range.  Each transaction would begin
by reading the current revision of each object.  If an invalidation
message arrives, it changes its strategy to read revisions of objects
that were valid before the transaction that generated the invalidation
committed.

This strategy doesn't eliminate the conflict errors on reads, but it
makes them much less likely to occur.  They could still occur because,
e.g., the storage doesn't keep revisions or the storage gets backed
during a long-running transaction and the pack deletes some of the
needed revisions.  

This strategy will most likely help with read-only transactions,
because a write will always attempt to write the latest revision of an
object.  If it tries to write object O4 and O4 was modified after the
transaction began, the write will fail.

I can't yet estimate when we would add the multi-version reads to
ZODB.  It's a non-trivial change.  It requires extra support for the
storage to reading an old revision of an object and it requires extra
machinery to determine what the time-range for the transaction is.  We
might need to do some of these changes for ReplicatedStorage, which
increases the chance that it will happen soon.

Jeremy