[Zope] More on understanding conflicts

[Dennis Allison]

Michael -- 

You raise an interesting point.  We use sessions to hold the
volatile state for each user in a highly interactive, highly personalized
site. A session lasts from login until logout or until there has been no
activity for a long time.  Perhaps, if I were to simply excise the session
timeout mechanism and the write of the last access time all would be well.  
We do not use session timeouts and could easily handle the recovery in a 
different way.

I have changed the WRITEGRANULARITY and other parameters to be more 
comensurate with our long session times.

I have begun to think that the Zope session variable mechanism, as
implemented, addresses a different part of the storage spectrum than we
need.  A small number of per user parameters (3 to 10) are used by almost
every request. Most of the parameters are set at user login and are not
changed for the duration of the session; sessions can last a long time, up
to ten hours, say, Some of the parameters change more rapidly, for
example, those that capture the current navigation state.  It may be that
we need two different kinds of session data, one for login-static data and
another for dynamic data.

I'm still working on getting some detailed traceback information,
difficult when the problems occur only infrequently on loaded production
systems with a 24x7 service profile.  We are moving to Zope 2.8.5 to take
advantage of the better conflict reporting.

On Thu, 22 Dec 2005, Michael Dunstan wrote:

> On 12/22/05, Dennis Allison <allison at shasta.stanford.edu> wrote:
> > The issue here has had to do with session variables and their iteraction
> > with the persistence mechanism and conflicts and multiple threads for the
> > same session.  Chris McDonough has pointed out that session variables can
> > cause writes and write conflicts even if the only Zope level access is a
> > read.  I thought MVCC works out of the box for Zope 2.8.4 which uses ZODB
> > 2.3.4. Am I wrong?
> >
> > The real surprise was that you can get a write conflict from a pair of
> > session variable reads!
> 
> The missing detail here is that reading a session object causes a
> write to the database to update the last access time for that session
> object. So what looks like a plain old read to the application code
> above, in-fact includes a write just below the surface. (This is a
> design choice so that stale session objects can be removed.)
> 
> There are implementation details of TransientObject that attempt to
> mitigate some of the pain:
> 
> - The access time has a WRITEGRANULARITY. That can help a bit by
> avoiding writing altogether some of the time. But still there is need
> to write (and so potential contention) each time a boundary of that
> granularity is crossed.
> 
> - And _p_resolveConflict of TransientObject will most often pick a
> state to avoid a conflict error. When it does there is the performance
> cost of writing to database but there will be no ConflictError.
> However, I've suggested in the past that you should comment out
> _p_resolveConflict while debugging consistency problems. What is the
> current state of your _p_resolveConflict now? _p_resolveConflict
> includes a DIWM'ly component. Perhaps you still need to avoid that
> component of _p_resolveConflict? In that case a _p_resolveConflict
> could be engineered that can resolve the case of just updates to the
> access time? But still bails out when presented with two states that
> have explicit updates to maintain consistency.
> 
> Though that's all just conjecture till we see your tracebacks for your
> conflicts.
> 
> michael
> 

--