[Zope] resolving conflict errors

[Dennis Allison]

On Fri, 14 Oct 2005, Chris McDonough wrote:

> Dennis Allison asked:

> > What data does Zope store in the ZODB when an object is evaluated?
> 
> None that you don't tell it to.  Typically conflict errors are a result
> of two threads calling code which changes the same object at the same
> time, but nothing that Zope does "under the hood" causes it; it is
> always caused by application code.
> 
> One "exception" to this rule is conflict errors raised when using Zope
> sessions.  It's not actually an exception to the rule, but programmers
> are shielded from the fact that sessions store data in ZODB when you use
> the session API (e.g. REQUEST.SESSION).  The sessioning machinery needs
> to manage housekeeping info whenever the API is used to expire old
> sessions and create new ones, so although it may not "look" like you are
> writing to the ZODB when you use sessions (even to read data out of
> them), you potentially are. 
> 
> Zope 2.8 has a ZODB that support multiversion concurrency control, which
> eliminates a certain class of conflict errors (read conflict errors), so
> if you are getting a lot of these, and you can get away with using 2.8,
> I'd suggest doing so.

The problem I am trying to resolve appears to be load related.  The
observed symptom is that (some) session variables spontaneously disappear.  
There appears to be some connection to conflicts, but the exact mechanism
and the relationship is not yet clear.

BTW, when I first began trying to resolve this problem, we were running 
Zope 2.7.6.  We moved to Zope 2.8,4 to take advantage of the later ZODB. 
That move was a good one, but it has cause some migration pain.

The session problem has been with us forever, at least since Zope 2.5.0.  
They are Heisenbugs and appear apparently randomly.  Detailed logs should 
help some, but so far have not brought joy.

So, I've been looking through the code trying to find places where some 
infrequent event could cause the problem.   

Chris pointed out that session variables can cause conflict errors (both
read-read and read-write)  when the session API is used.  I've been trying
to explore that interface and have not yet found all the pieces.  Some
hints to the reader would help.

I see where HTTPRequest manages the Zope user interface to the
session variables which includes a mechanism for lazy access--the
dictionary _lazies provides a list of callables.  When a variable is
accessed, it is promoted to Request by executing the callable and storing
the value.  The _lazies entry corresponding to the session variable is
deleted.  What I have not been able to find is where this is maintained
in a persistent fashion.  Can someone provide a pointer?