[Zope] database conflicts and the _p_oid missing attribute bug

[Dennis Allison]

Jeremy & List --
I have been having problems with Zope locking up (deadlock?) when 
ZODB conflicts occur.  I am running Zope 2.6.2b3 with the 3.1.2b2
version of the ZODB (that is, a vanilla install).  Python is 2.1.3.

One of the failing methods, QLOG, that is a Python Script that computes 
some parameters (including calling an External Method to create 
a unique id) and then invokes a ZSQLMethod to insert information into a 
MySQL database.

The other failing method is the logged_in element of the commonly used 
Cookie Crumbler authentication system.  It does a bit of Javascript setup 
and then invokes a Python Script that creates a bunch of session variables 
which it populates from MySQL (4 queries) and writes a row into a MySQL 
table.  Each of the sql queries calls a PythonScript as part of the 
dynamic expansion of the query. The final query calls an External Method 
to get a unique identifier.

The diagnostic is always:

2004-01-22T12:25:15 ERROR(200) ZODB A storage error occured during 
transaction abort.  This shouldn't happen.
Traceback (innermost last):
  File /home/zopesys/lib/python/ZODB/Transaction.py, line 262, in commit
  File /home/zopesys/lib/python/ZODB/Transaction.py, line 393, in 
_commit_error
AttributeError: Surrogate instance has no attribute '_p_oid'

As I understand what's happening (after reading the logs carefully), I
have one or more folks trying to use the same object.  Zope's ZODB
serializes the access and aborts all but the oldest of the conflicting
transactions.  The diagnostic occurs because a registered object is 
missing the _p_oid attribute that's used to rollback the transaction.  The 
big question is which of the several sub-transactions is the culprit--or 
is it a bug as Dieter has suggested, soluable by providing a default 
value for _p_oid -- 

Dieter wrote:

    You can fix this by substituting "getattr(o,'_p_oid','<not
    persistent>')" for "o._p_oid" in "_commit_error".

I've been a bit unwilling to modify this part of Zope until I fully 
understand the problem.  How can I find the offending object(s)?  Why 
don't they have the proper _p_oid attribute?  Is this truly a bug?

And, on another front, it looks like conflicts of the sort that are 
triggering the bug are going to be relatively common.  They are real and 
are ascerbated by the time required for database, file, and computation.
Would installing ZEO help minimize conflicts?



-- 
Dennis Allison * Computer Systems Laboratory * Gates 227
               * Stanford University *  Stanford CA  94305
	       * (650) 723-9213 * (650) 723-0033 fax
	       * allison at shasta.stanford.edu
	       * allison at sumeru.stanford.edu