[ZODB-Dev] conflict resolution for PersistentList

[Chris McDonough]

On Mon, 2004-05-24 at 09:20, Diez B. Roggisch wrote:
> >
> > Are you sure that the conflicts you care about are in this parent
> > object?  Without knowing anything about your application logic, I would
> > guess that items is modified more often than its parent.  If so, you
> > need an _p_resolveConflict() method on items, not on the parent.  You
> > could accomplish that by subclassing PersistentList and adding the
> > _p_resolveConflict method.
> 
> The problem is that the list of items itself creates the conflict - in my 
> older post I said that the list of items in parent is modified concurrently - 
> new items are created and appended to the list. Then the conflict arises 
> inside the list itself. 

The fact that the conflict resolution handler is called on the parent
object indicates that the conflict is not happening (at least in your
example) as a result of concurrent mutation of the PersistentList
itself.  Instead it means that the conflict is happening due to two
threads concurrently changing/reading the state of the parent of the
PersistentList.

> Having a _p_resolveConflict on UserList didn't help me as it doesn't get 
> called, while having one at Parent gets called but contains the pickled 
> version as state - nothing I can work with.

I think this just means that the conflict isn't happening where you
believe it's happening.  If the conflict is happening as a result of
concurrent access to the parent, you'll need to write a conflict
resolution handler on the parent.  If you write a simple
_p_resolveConflict method that prints the three states and attach it to
the parent class, what gets shown during a conflict?

Have you confirmed that your ZODB version has the "suppressed conflicts
prevent commit" bug fix in it as per Tim's suggestion?:

"""
The easiest way to check is probably to look in ZODB/tests/testZODB.py,
and see whether this test function exists:

    def checkReadConflictIgnored(self):
        # Test that an application that catches a read conflict and
        # continues can not commit the transaction later.
        ...
"""

If not, a good place to start is by getting a ZODB version that has this
feature.  That can definitely be done by grabbing ZODB out of the Zope
2.7 CVS branch, and perhaps by getting a non-Zope ZODB 3.2 release
(according to what Tim and Jeremy are saying in this thread).

> Maybe the solution to this problem is to use a mapping as storage, not a list, 
> and a synchronized key generator - bet what happens when two mappings like 
> this:
> 
> {0: OldItem, 1:OldItem, 2: NewItem, 4: NewItem}
> {0: OldItem, 1:OldItem, 3: NewItem, 5: NewItem}
> 
> are commited after each other? OldItem shall denote that the item has been in 
> the mapping before the current tratnsaction.

If they are simple dictionaries, a conflict error will be raised
(regardless of the three states).

Various "BTrees" data structures also provide mapping-like features, and
they provide a bit of built-in conflict resolution, which is based on a
hardcoded policy.  But what actually would happen if these items were
BTree-variant objects depends on the state stored in the database when
these two transactions commit simultaneously (I couldn't tell you what I
think the actual effect would be without seeing the original database
state).  In any case, a set of rules will be consulted.  These are
documented in the BTrees code, and also outlined by Tim at
http://mail.zope.org/pipermail/zodb-dev/2004-February/006768.html .  I
think he also added this to some set of semi-official ZODB docs
someplace, but I can't find that at the moment.

- C