[ZODB-Dev] Debugging ConflictErrors

[Christian Robottom Reis]

On Thu, Jan 15, 2004 at 02:47:21PM +0100, Thomas Güttler wrote:
> Up to now I log ConflictErrors like this:
> 
>         # First version taken from Dulcinea/lib/ui/publish.py
>         retries=15
>         for i in range(retries):
>             try:
>                 return Publisher.try_publish(self, request, path)
>             except ConflictError, exc:
>                 obj=self.root_namespace._p_jar[exc.oid]
>                 print "ConflictError %sv%s: retrying request. " \
>                       "path=%s user=%s obj=%s oid=%s" % (
>                     i, retries,
>                     path, os.environ.get("REMOTE_USER", ""), repr(exc.oid),
>                     obj)
>                 get_transaction().abort()
>                 request.response = HTTPResponse() # reset response object
>                 time.sleep(2 + (random.random() * (i+1)))
>         else:
>             raise RuntimeError("too many conflict errors")
> 
> The conflicts always are in my root object. This is
> the object which holds all other objects. I use ZODB 3.2

I'm not sure I've said this before, but using the root object to hold a
large/variable number of objects is a Bad Idea. The root object is a
PersistentMapping, which provides no conflict isolation, and in the long
run it's almost impossible to do any write-intensive concurrent work
with one without generating an endless number of conflicts.

A better pattern is to use the root PersistentMapping to hold other
containers that are better suited to the purpose of holding large
numbers of objects -- TreeSets and BTrees are what I would recommend.
The only thing to watch out is for their semantics (uniqueness for sets,
and totally-orderable btree keys). This will probably reduce the
incidence of conflicts significantly -- perhaps significantly enough to
make logging them a non-issue for your application <wink>.

Take care,
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 261 2331