[Zope-dev] Zope3 sessions and database conflicts

[Christian Theune]

On 03/04/2010 11:08 PM, Ross Patterson wrote:
> Hermann Himmelbauer <dusty at qwer.tk> writes:
> 
>> Hi,
>> For quite some time I see messages like this in my z3.log:
>>
>> 2010-03-02T16:27:14 WARNING ZopePublication Competing writes/reads 
>> at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/images/sponsor_logo.png: 
>> database conflict error (oid 0x063f, class BTrees.OOBTree.OOBucket, serial 
>> this txn started with 0x038484dc7d5ac044 2010-03-02 14:52:29.379960, serial 
>> currently committed 0x038484ff3c6b5455 2010-03-02 15:27:14.160763)
> 
> I note the conflict happened when the request was just for an image.
> I've seen this before under Zope2 when a PAS plugin was mis-using the
> session machinery as a cache during authentication of the request.  When
> ever a session object is accessed it initiates a write transaction in
> the ZODB.  With most, if not all, authentication methods, when a user is
> logged-in the auth tokens are included in every request which means
> every request from the logged-in user invokes the authentication
> machinery, including requests for images.  Since every page load
> involves multiple requests for page resources, the database gets
> overwhelmed with write transactions which inevitably conflict.  When the
> write conflict occurs, the publisher appropriately retries the request
> which multiplies the number of requests which multiplies the load which
> increases the amount of time taken in processing requests which
> multiplies the likelihood of write conflicts and your off to the races
> in the wrong direction.
> 
> So I'd suggest you find out what in a request for a simple resource
> (images, non-dynamic CSS or JS) is initiating the write transaction by
> invoking sessions.  It's likely there's an inappropriate use of sessions
> there.

I remember one thing about sessions where just the access to the session
would create a new session even if it was never touched. The API
actually provides two invokations: one to get a session and create if
not existing, the other got get a session and raise a KeyError instead
of creating a new one.


-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development