[Zope] resolving conflict errors

[Chris McDonough]

On Fri, 2005-10-14 at 09:27 -0700, Dennis Allison wrote:
> Zope 2.7.6
> 
> I am a bit confused.  
> 
> I have a Zope DTML method that is generating ZODB conflict errors.
> 
> The DTML method identified as producing the conflicts is a list of calls
> to other methods, conditionally executed.
> 
> Most conflicts don't cause problems because the backoff and restart of the 
> initial transaction will not have changed global state.  In our particular
> case, the conflicting transaction has changed global state in our RDBMS so
> when it gets rerun, some RDBMS transactions are duplicated.  And that's a 
> problem.  The solution, of course, is to resolve the conflicts properly.

Another solution is to use a RDBMS that fully supports transactions.
This is almost always the easiest solution.  ConflictErrors are a fact
of life if you use ZODB to store data; they are impossible to eliminate
entirely so no matter what, if you need to communicate with external
data stores that aren't transactional (MyISAM tables, LDAP, sendmail,
etc.) you need to anticipate duplicated requests in the code that
communicates with these systems.  For example, Jens Vagelpohl has a
replacement for Zope's MailHost that prevents retried requests due to
conflict errors from causing a mail to be resent.  I've written payment
systems that anticipate the fact that the request may be retried, so
instead of submitting the payment request twice, the code keeps around a
little cache about what it did "last" so it doesn't do it again.  And so
on.

> The first question: what data is generating the conflict?  

I believe that if you run Zope event logging at BLATHER level, the
traceback of every ConflictError exception is logged, which can give you
an idea of what is causing the errors.

> The DTML code 
> and all the method references are static and unchanged.   What data does
> Zope store in the ZODB when an object is evaluated? 

None that you don't tell it to.  Typically conflict errors are a result
of two threads calling code which changes the same object at the same
time, but nothing that Zope does "under the hood" causes it; it is
always caused by application code.

One "exception" to this rule is conflict errors raised when using Zope
sessions.  It's not actually an exception to the rule, but programmers
are shielded from the fact that sessions store data in ZODB when you use
the session API (e.g. REQUEST.SESSION).  The sessioning machinery needs
to manage housekeeping info whenever the API is used to expire old
sessions and create new ones, so although it may not "look" like you are
writing to the ZODB when you use sessions (even to read data out of
them), you potentially are. 

Zope 2.8 has a ZODB that support multiversion concurrency control, which
eliminates a certain class of conflict errors (read conflict errors), so
if you are getting a lot of these, and you can get away with using 2.8,
I'd suggest doing so.

> Presumably conflicts can be reolved programatically by setting a method 
> on the object
> 
>    _p_resolveConflict( self, old, saved, new )
> 
> and returning one or another of the states (old, saved, new).  	It's not 
> real clear how to do it.   

There are examples of "real-world" conflict resolution using this
mechanism in the Transience product included in Zope's
lib/python/Products.

- C