[Zope] resolving conflict errors

[Dennis Allison]

Thanks Chris.  On point as usual!

I was unaware that the session mechanism used the ZODB although a bit of
thought says it has to.  We don't store any data into the ZODB in these
methods, but we do use the session mechanism heavily.

I suppose that moving to a fully transactional database system would be 
the simplest solution, but I'm a bit wary of doing so on a live system.



On Fri, 14 Oct 2005, Chris McDonough wrote:

> On Fri, 2005-10-14 at 09:27 -0700, Dennis Allison wrote:
> > Zope 2.7.6
> > 
> > I am a bit confused.  
> > 
> > I have a Zope DTML method that is generating ZODB conflict errors.
> > 
> > The DTML method identified as producing the conflicts is a list of calls
> > to other methods, conditionally executed.
> > 
> > Most conflicts don't cause problems because the backoff and restart of the 
> > initial transaction will not have changed global state.  In our particular
> > case, the conflicting transaction has changed global state in our RDBMS so
> > when it gets rerun, some RDBMS transactions are duplicated.  And that's a 
> > problem.  The solution, of course, is to resolve the conflicts properly.
> 
> Another solution is to use a RDBMS that fully supports transactions.
> This is almost always the easiest solution.  ConflictErrors are a fact
> of life if you use ZODB to store data; they are impossible to eliminate
> entirely so no matter what, if you need to communicate with external
> data stores that aren't transactional (MyISAM tables, LDAP, sendmail,
> etc.) you need to anticipate duplicated requests in the code that
> communicates with these systems.  For example, Jens Vagelpohl has a
> replacement for Zope's MailHost that prevents retried requests due to
> conflict errors from causing a mail to be resent.  I've written payment
> systems that anticipate the fact that the request may be retried, so
> instead of submitting the payment request twice, the code keeps around a
> little cache about what it did "last" so it doesn't do it again.  And so
> on.
> 
> > The first question: what data is generating the conflict?  
> 
> I believe that if you run Zope event logging at BLATHER level, the
> traceback of every ConflictError exception is logged, which can give you
> an idea of what is causing the errors.
> 
> > The DTML code 
> > and all the method references are static and unchanged.   What data does
> > Zope store in the ZODB when an object is evaluated? 
> 
> None that you don't tell it to.  Typically conflict errors are a result
> of two threads calling code which changes the same object at the same
> time, but nothing that Zope does "under the hood" causes it; it is
> always caused by application code.
> 
> One "exception" to this rule is conflict errors raised when using Zope
> sessions.  It's not actually an exception to the rule, but programmers
> are shielded from the fact that sessions store data in ZODB when you use
> the session API (e.g. REQUEST.SESSION).  The sessioning machinery needs
> to manage housekeeping info whenever the API is used to expire old
> sessions and create new ones, so although it may not "look" like you are
> writing to the ZODB when you use sessions (even to read data out of
> them), you potentially are. 
> 
> Zope 2.8 has a ZODB that support multiversion concurrency control, which
> eliminates a certain class of conflict errors (read conflict errors), so
> if you are getting a lot of these, and you can get away with using 2.8,
> I'd suggest doing so.
> 
> > Presumably conflicts can be reolved programatically by setting a method 
> > on the object
> > 
> >    _p_resolveConflict( self, old, saved, new )
> > 
> > and returning one or another of the states (old, saved, new).  	It's not 
> > real clear how to do it.   
> 
> There are examples of "real-world" conflict resolution using this
> mechanism in the Transience product included in Zope's
> lib/python/Products.
> 
> - C
> 
> 

--