[Zope] session variables in the presence of conflicts

Dennis Allison allison at shasta.stanford.edu
Sun Nov 13 04:39:40 EST 2005


Zope 2.8.4
ZEO 3.4.2
ZODB 3.4.2
Python 2.4.2 or 2.3.5 
MySQL 4.0.20
MySQL-Python 1.2.0
MYSQLDA 2.0.9

We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but 
the ability to avoid read conflicts under ZODB 3.4.2.  We have been having 
a lot of problems: more conflict errors, release of unreleased locks in 
MySQL transactions, loss of session variables, and others.

We use session variables and MySQL very heavily.  Our MySQL tables are
MyISAM and are not transactional.

ZODB3 3.4.1 introduce some changes to the conflict resolution which
appears to be a logical improvement but also appears to have acerbated
our problems.  In the ZODB# 3.4.2 system, if a commit fails, an exception
is raised and the transaction is aborted.  Zope then attempts to reschedule
the transaction.  The abort needs to roll back MySQL transactions that are
part of the conflicting ZODB and the  Session Variable Store so the 
restart is clean.

In our case, we have been using MyISAM tables, which are not
transactional.  The rollbacks fail and the locking in the database adapter
gets confused. Changing the table type from MyISAM to Innodb, which is
transactional, should resolve the MySQL problems, although it has yet to
be tested.

Conflict errors also arise through the session variables.  Since we have a
multi-Zope system configuration sharing a single ZEO, the session
variables are stored in a Temporary Storage object on the ZEO host.  When
a conflict occurs, a number of things, none good, apparently can happen.
Session variable transactions cannot be rolled back (as the temporary
storage mechanism is not transactional).  And, at a different level,
session variable persistence requires that transactions require a
copy-out/copy-in mechanism to identify those changes which have been
changed and need to persisted; an aborted transaction may not be marked as
persistent and so will disappear at some apparently random time in the
future when the temporary store is packed and cruft removed.

Some refactoring of the programming can help minimize the potential for 
session variable conflicts, but they cannot be eliminated.

If systems stability in the face of session variable conflicts is 
important, 

  1.  Does the store for session variable need to be transactional 
  and support rollbacks?

and 

  2.  What Temporary Storage object provides the right mechanism to 
  manage session variables with rollback efficiently?





More information about the Zope mailing list