[Zope] How to handle ZODB conflict errors

[Chris McDonough]

This is good info.  I've posted this at
http://www.zopewiki.org/ConflictErrors so I can find it to refer people
to again. ;-)

On Wed, 2004-09-29 at 09:54, Sean Hastings wrote:
> The slowdowns result when various transactions are trying to modify the same
> data. When a conflict happens, one of the transactions is rolled back and
> attempted again from the beginning. If the transaction takes a long time to
> execute, and this happens many times, the delays can be considerable. Only
> after some number of attempts, with continuing conflicts occurring, will the
> transaction give up and allow the Conflict Error to actually be raised back
> to the level of producing an error page, but each one will be logged in your
> event.log file.
> 
> I wrote the following to explain conflicts to my own satisfaction:
> -------------------------------------------------------------------
> 
> There are two types of conflict errors:
> 
> Read Conflict:
> --------------
> Transaction 1 reads object A.
> Transaction 2 modifies object B.
> Transaction 1 attempts to read object B, but notices that object B has been
> modified since Transaction 1 began to read data. Since the data it read from
> A, and the data that it now finds in B are not necessarily consistent, a
> Read Conflict Error is produced.
> Transaction 1 rolls back and restarts.
> 
> 
> Write Conflict:
> ---------------
> Transaction 1 reads object A.
> Transaction 2 modifies object A.
> Transaction 1 attempts to modify object A, but notices that object A has
> been modified since it was read. Since overwriting the changes made by
> Transaction 2 would cause "lost update problems" and possible data
> inconsistency, a Write Conflict Error is produced.
> If conflict is not handled (see below), Transaction 1 rolls back and
> restarts.
> 
> 
> Solutions:
> ---------
> 
> 
> Read Conflicts - If you do not care about consistency during your read, you
> can execute the statement "get_transaction().commit()" after each new object
> accessed. This breaks your transactions up into smaller transactions that
> are MUCH less likely to cause conflicts. I understand that Zope 2.8 (alpha
> any day now) will eliminate Read Conflicts by including the ability to read
> old constant states. In the example above, Transaction 1 would then just
> read the old state of B before Transaction 2 had modified it.
> 
> 
> Write Conflicts - You can define a method on any ZODB object called
> "_p_resolveConflict". When a write conflict occurs, the first thing that
> happens is that Zope tries to call this method to handle the problem,
> passing it all the possible states of the object: OLD, FOUND, and TRIED. OLD
> is the state that the transaction first read. FOUND is the new modified
> state that some other transaction saved the object as. TRIED is the state
> that the transaction wanted to save the object as when it realized that
> there was a conflict. This method is intended to return a merged version of
> the two objects. If you do not care about lost updates, this method could be
> as simple as:
> 
>         def _p_resolveConflict(self,old,found,tried):
>                 return tried
> 
> -------------------------------------------------------------------
> 
> Here is an example of some code that I think works to handle write conflicts
> in an order independent list. The object in question is a ZODB wrapper for a
> list. The [] is stored in the "data" element referred to in saved.data,
> old.data, etc:
> 
> 	def _p_resolveConflict(self,old,saved,new):
> 		"""
> 		Merges two possible list states
> 		Accounts only for additions and deletions, not changes of order
> 		Prevents duplicate add or remove of same Item during conflict
> 		"""
> 
> 		#get changes made in saved state
> 		saved_added = []
> 		saved_removed = []
> 		for item in saved.data:
> 			for i in range(saved.count(item) - old.count(item)):
> 				saved_added.append(item)
> 		for item in old.data:
> 			for i in range(old.count(item) - saved.count(item)):
> 				saved_removed.append(item)
> 
> 		#get changes made in new state
> 		new_added = []
> 		new_removed = []
> 		for item in new.data:
> 			for i in range(new.count(item) - old.count(item)):
> 				saved_added.append(item)
> 		for item in old.data:
> 			for i in range(old.count(item) - new.count(item)):
> 				new_removed.append(item)
> 
> 		#get duplicate changes
> 		both_added=[]
> 		for item in new_added:
> 			if saved_added.count(item):
> 				both_added.append(item)
> 				new_added.remove(item)
> 				saved_added.remove(item)
> 		both_removed=[]
> 		for item in new_removed:
> 			if saved_removed.count(item):
> 				both_removed.append(item)
> 				new_removed.remove(item)
> 				saved_removed.remove(item)
> 
> 		#apply changes
> 		old.extend(saved_added)
> 		old.extend(new_added)
> 		old.extend(both_added)
> 		for item in saved_removed:
> 			old.remove(item)
> 		for item in new_removed:
> 			old.remove(item)
> 		for item in both_removed:
> 			old.remove(item)
> 
> 		return old
> 
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
>