[Zope] session variables in the presence of conflicts

[Peter Bengtsson]

Have you considered upgrading to PostgreSQL? Things might have changed
since the last time I tried mysql but I'm pretty sure Postgresql and
psycopg deals much better with transactions in Zope. Admittedly, this
doesn't explain why you're getting into trouble just because you've
upgraded to zodb 3.4.2

On 11/13/05, Dennis Allison <allison at shasta.stanford.edu> wrote:
> Zope 2.8.4
> ZEO 3.4.2
> ZODB 3.4.2
> Python 2.4.2 or 2.3.5
> MySQL 4.0.20
> MySQL-Python 1.2.0
> MYSQLDA 2.0.9
>
> We have just moved from Zope 2.7.6 to Zope 2.8.4 motivated, in part, but
> the ability to avoid read conflicts under ZODB3.4.2.  We have been having
> a lot of problems: more conflict errors, release of unreleased locks in
> MySQL transactions, loss of session variables, and others.
>
> We use session variables and MySQL very heavily.  Our MySQL tables are
> MyISAM and are not transactional.
>
> ZODB3 3.4.1 introduce some changes to the conflict resolution which
> appears to be a logical improvement but also appears to have acerbated
> our problems.  In the ZODB# 3.4.2 system, if a commit fails, an exception
> is raised and the transaction is aborted.  Zope then attempts to reschedule
> the transaction.  The abort needs to roll back MySQL transactions that are
> part of the conflicting ZODB and the  Session Variable Store so the
> restart is clean.
>
> In our case, we have been using MyISAM tables, which are not
> transactional.  The rollbacks fail and the locking in the database adapter
> gets confused. Changing the table type from MyISAM to Innodb, which is
> transactional, should resolve the MySQL problems, although it has yet to
> be tested.
>
> Conflict errors also arise through the session variables.  Since we have a
> multi-Zope system configuration sharing a single ZEO, the session
> variables are stored in a Temporary Storage object on the ZEO host.  When
> a conflict occurs, a number of things, none good, apparently can happen.
> Session variable transactions cannot be rolled back (as the temporary
> storage mechanism is not transactional).  And, at a different level,
> session variable persistence requires that transactions require a
> copy-out/copy-in mechanism to identify those changes which have been
> changed and need to persisted; an aborted transaction may not be marked as
> persistent and so will disappear at some apparently random time in the
> future when the temporary store is packed and cruft removed.
>
> Some refactoring of the programming can help minimize the potential for
> session variable conflicts, but they cannot be eliminated.
>
> If systems stability in the face of session variable conflicts is
> important,
>
>   1.  Does the store for session variable need to be transactional
>   and support rollbacks?
>
> and
>
>   2.  What Temporary Storage object provides the right mechanism to
>   manage session variables with rollback efficiently?
>
>
>
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
>


--
Peter Bengtsson,
work www.fry-it.com
home www.peterbe.com
hobby www.issuetrackerproduct.com