[Zope-dev] session concurrency

[Dieter Maurer]

Santi Camps wrote at 2003-11-27 17:41 +0100:
 > ...
 > One thing is true:  Changing session object simultaniously in various
 > frames doesn't work fine.  Sometimes session data is not the espected
 > becouse another frame has erased your changes in the session object.  
 > 
 > I think this is a serious inconvenient in front of other platforms. 

This is a general problem of concurrently updating the same object.

Zope's "Session" mechanism is not that bad in this respect (it is
in some way better than that for other plattforms).
When it works as advertised (I see it working this way; you do not),
then concurrent modification will result in a "ConflictError"
with automatic request retrial. This provides for consistency
despite concurrent updates.

A *BIG* problem is the large granularity of a Zope session.
It is essentially one big monolitic object (with respect to
concurrent update). This makes "ConflictError" much more likely.

I have never seen disappearing sessions -- but I must admit
that we use our own session implementation - deriving from
Zope's session but getting rid of the over-complex session
management code.
This code caused many "inconsistency" errors
for the session management data structures; which is a bad sign.
The complexity comes from the fact that it tries to scale well
for large numbers of sessions. Our simplified code scales not
that well but can easily cope with several thousand (locally maintained)
sessions we sometimes see.

 > >From my point of view, it would be better to have SESSION out of the
 > transaction but with changes stored inmediatly.  You don't?

No: you would face the general inconsistency problem caused by
concurrent updates of an object. Analysing and fixing such
problems is a nightmare (they occur non-deterministically, they are extremely
difficult to reproduce and analyse).

 > Perhaps some day I will be able to implement this for myself.

No need. There is a product called "ZSession".
It maintains session data in RAM and (I think) allows threads
to read/write sessions asynchronously.
It is probably no longer maintained, now that Zope comes
with Sessions in the core distribution.
But that need not be a big problem. Should it break in some
future Zope version (which may be possible), the fixes should
be quite easy and straight forward.

 > Until
 > that, do you think I should enter a bug, or this cann't be considered as
 > a bug?

When you see session objects disappearing (and they have not expired),
then this is a bug!

As said: The advertised behaviour when several requests modify
the same session object is that all but the first committing request
get a "ConflictError" and are retried. Everything else is a bug.


Another note: it is known (though we do not yet fully understand why) that
sessions behave non-deterministically (disappear, revert to old values)
when they are read/written in "standard_error_message" (Zope's
error handling hook for failing requests).

"standard_error_message" is called after the transaction has been
aborted -- thus any (transactional) effect from the request
has already been wiped out. Accessing "Session" there may read old
values or create a new (empty) session.

Apparently, and this is what we do not yet understand, such phony
sessions seem to be able to spill over into later requests.
One hypothesis was that it were caused by Zope forgetting to
abort/commit the transaction implicitly created for "standard_error_message"
execution (this definitely is a bug!). But the transaction
*is* aborted later, before it is being used in another request -- maybe
this happens then in the wrong context?

I will not see problems caused by this
"failing to commit/abort error transaction" bug because
I fixed our Zope as soon as the bug became known.

I am not sure whether the bug is fixed in Zope 2.6.2.
That's a disadvantage of maintaining our own Zope version - I do
not know and can not easily check the distributed code any longer.

-- 
Dieter