[ZODB-Dev] conflict resolution for PersistentList

[Tim Peters]

[Tim Peters]
>>> But you still shouldn't have seen *inconsistent* data.  That's another
>>> mystery here.

[Diez Roggisch]
>> I don't have inconsistent data - its just that the thread that committed
>> first has the list of childs written, while the second ones data is lost.
>> First that went unnoticed due to the broken exception handling.

[Chris McDonough]
> It's good you were able to work around this, but it still shouldn't
> happen.  Your data actually *is* inconsistent, given the constraints
> that later versions of ZODB places on commits after a read conflict is
> raised, even if it is caught later.

I don't think we can know that -- there's too much guessing here.

Take a look at checkReadConflictIgnored.  The heart of the matter is that
the penultimate line doesn't blow up (meaning it *does* raise
ConflictError):

        self.assertRaises(ConflictError, cn2.getTransaction().commit)

However, after the ConflictError raised by the *attempt* to commit has been
suppressed, that transaction is history, and a new transaction begins (which
is implicit, which makes it devilishly obscure).  Concretely:  if you
replace the line above by these 5:

        self.assert_(not real_data2.has_key("a"))
        self.assertRaises(ConflictError, cn2.getTransaction().commit)
        self.assert_(real_data2.has_key("a"))
        self.assert_(not real_data2._p_changed)
        cn2.getTransaction().commit()

the test still passes.  Suppressing the error on the first attempt to commit
effectively wipes out (undoes) the changes we were trying to commit, and
silently refetches fresh data (and the final line there doesn't actually
commit any changes).  It's as if you had done an explicit transaction
abort() immediately following the suppressed commit() exception.

That's consistent with one transaction winning, and the other transaction's
changes vanishing silently.  What I can't guess is exactly where and how
Diez's exception-handling was overly embracing (e.g., don't know whether
errors were suppressed only during commit() attempts, only prior to commit()
attempts, or were suppressed in both places).

> I'm paranoid because I have an application that would break badly if
> this constraint wasn't met and I want to make sure that indeed you are
> using a version of ZODB that was known to have this bug.

Yup, that's still an open question.  My bet is that he was using a more
recent ZODB.

> So when you send your code, could you also tell us which version of ZODB
> you're using and confirm that it does or does not have the test method
> that Tim and Jeremy confirmed to be a good indicator about whether the
> "prevent commit after read conflict" behavior is present?

That would be good to know.