Casey, I am pretty sure I understand the idea of a "write Conflict" in Zope. This is what I have been understanding: a.) Transaction 1 reads Object A b.) Transaction 2 reads Object A c.) Transaction 1 changes Object A to create Object A1 d.) Transaction 2 changes Object A to create Object A2 e.) Transaction 1 Commits Changes - saves A1 as readable state f.) Transaction 2 Attempts to Commit - raises WriteConflict g.) A._p_resolveConflict(A,A1,A2) is called to resolve h.) Resolved state is returned and saved as new readable state Else Transaction 2 is rolled back and restarted Is this a Correct description of a write Conflict? How is a ReadConflict Different? --Sean
Message: 10 Date: Fri, 23 Jan 2004 11:36:24 -0500 From: Casey Duncan <casey@zope.com> Subject: Re: [Zope] Re: database conflicts and the _p_oid missing attribute bug To: srl@boston.com Cc: jeremy@zope.com, zope@zope.org, allison@shasta.stanford.edu Message-ID: <20040123113624.680d6469.casey@zope.com> Content-Type: text/plain; charset=US-ASCII
On Fri, 23 Jan 2004 09:56:23 -0500 Shane Landrum <srl@boston.com> wrote:
On Thu, Jan 22, 2004 at 11:26:23PM -0500, Jeremy Hylton wrote:
That right in a rough sense. ZODB uses optimistic concurrency control, so later transactions are aborted if they conflict with already-committed ones. But the transactions actually run concurrently.
Could you explain this a bit more? Because we run relatively high-write operations on our Zope/ZEO setup, we hit ConflictErrors quite a bit. I'd like to understand the underlying machinery here, ideally so I can come up with a fix.
Here is a simple example:
Suppose you have an object 'somefile' that has an attribute 'data'. Now suppose that there are two concurrent processes modifying 'somefile':
Process 1: somefile.data = 'Mary had a little lamb'
Process 2: somefile.data = 'The Quick Brown Fox...'
The concurrency control is "optimistic" because it allows both processes to change the same file object (there are no locks).
Now suppose that process 1 commits the transaction. This saves 'Mary had a little lamb' as the new file data in the ZODB. Inevitably, process 2 also commits its transaction. The ZODB sees that the file was modified since process 2's transaction began, and this is a conflict (specifically a write conflict).
When the ZODB detects a write conflict it sees if the object supports conflict resolution (via a special method). If so, the method is called to allow the object to try to resolve the conflict. Most objects do not have conflict resolution, however. In this case ZODB raises a ConflictError (thereby aborting the transaction). This error is caught by Zope. In response, Zope retries the web request which replays the transaction again. If it gets another ConflictError is tries again (up to 3 times). After 3 times it gives up and returns an error to the user.
In most cases, the first retry is sufficent. More retries are necessary as the system becomes busier (more concurrency). This causes extra work since many transactions need to be retried. This can exacerbate the problem and cause performance degradation.
Conflicts tend to center around "hotspots", objects that are changed often by many requests. The Catalog is a classic example since it tends to get changed everytime new content is added to the system. Content repository folders are another example.
Both of these "hotspots" have their own conflict resolution code (they typically use BTree objects which can resolve many write conflicts internally). Write conflicts tend to be pretty manageable if you recognize hotspots and use the proper data types (like BTrees) or add your own conflict resolution (not for the faint of heart, however).
The more insidious kind of conflicts are Read Conflicts. They are caused (in simple terms) when one transaction changes an object that another concurrent transaction is about to use (These are called "dirty reads"). You can't resolve these kinds of conflicts and they can be very tricky to prevent especially in a busy system. Luckily ZODB 3.3 has a new feature: "Multi-version concurrency control" (MVCC) which resolves read conflicts (or at worst changes them to write conflicts). Once ZODB 3.3 gets released in Zope, we can look forward to many fewer conflict errors.
-Casey
------------------------------
_______________________________________________ Zope maillist - Zope@zope.org
End of Zope Digest, Vol 6, Issue 38 ***********************************