[Zope-dev] I volunteer to make ConflictError handling "better" if you guys can help me ;-)

[Chris Withers]

Dieter Maurer wrote:
>>What is this telling me?
> 
> It is incredibly stupid.

So I'm gathering...

> The message above only tells you, that (at the given time)
> a request for "/some_uri" resulted in a "ConflictError"
> and that since startup (at the given time) 347 conflicts occured.
> 
> Unfortunately, it does not tell you
> 
>   *  what object caused the conflict
> 
>   *  whether it is a read or a write conflict
>      (read conflicts are very rare since MVCC introduction,
>      but they may still happen)
> 
>   *  for write conflicts: what versions of the object did particate

...and most importantly, whether or not this conflict error was 
resolved! (ie: did the user see this as an error, or just poor performance?)

> A long time ago, I posted an
> extension making this additional information available (it
> is all present in the exception instance. Zope is just too stupid
> to read (and log) it).

I would really appreciate it if you could dig this out and give me a 
direct url. If you do, I promise I will see it gets properly merged into 
the Zope core...

>>Did the user actually see a ConflictError page?
> 
> Usually not.

...which, as I suspect you agree with me, simple isn't a good enough 
answer ;-)

>>I analysed the logs from our cluster and we're getting about 450 
>>conflict errors in our busiest hours when the cluster of 8 ZEO clients 
>>is taking about 11,000 hits in that hour.
>>
>>Is this 'bad'?
> 
> I would not be happy: it is about 5 %.

Yeah, me neither, but it would be great to quantify all of this :-)

>>If so, where should I start to make things better?
> 
> You find out which objects cause the conflicts.
> 
> You analyse what you can do to reduce concurrent writes
> to these objects (split into separate persistent subobjects)
> or whether you can provide conflict resolution.

...and I think the best way to get cracking with this is to improve 
logging and tidy up the retrying code for ConflictError. I've 
volunteered to do that, and will get to it asap, but I'm gonna need as 
much help as possible, particularly in reviewing code.

OK, I'm guessing I'll cut a branch when I start, I really want this to 
land ASAP. Where should I start from? Will there be more 2.8 releases or 
should I go for 2.9 and then merge to the trunk?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk