[Zope-dev] Conflict Errors

[Chris McDonough]

> Hurm, well, it appears to happen when we're doing a lot of sequential
> write to a product (automated bulk data upload), but the thing that
> triggers it off is actually trying to read a page from the same area of
> the site... confusing. Although that could also be a one-off
> coincidence...

Conflict errors can be raised by three things:  the storage, the
transaction, and the connection.  The storage will only raise the conflict
error on a write, this is for sure.  The transaction and the connection
objects I looked at a little today, and it's unclear .. although as I
understand it, the idea of a ConflictError is to prevent the modification of
an object by two threads at the same time, and should have nothing to do
with reads.

> You say no corruption or anything, but if a submit results in that
> error, does the submitted form data get processed or not?

No... the transaction is aborted...

> > Careful application coding can reduce the chance of conflict errors.
>
> Can you describe what you mean by careful application coding?

Ah I knew I wasn't gonna get away with that one.  ;-)  I probably shoulda
kept my mouth shut.

If you're writing a ZODB-based app that uses multiple database connections
(like Zope ;-), you should take care to stagger high-load writes against an
object in the threads that use those database connections.  The ZODB tries
hard to avoid error conditions for you by retrying conflicting writes, but
if you really pound the snot out of a particular object, it'll let you know.

That said, I doubt I can be more specific, because to be honest I don't know
the *real* answer myself.  Sigh.  I think I need to learn more about
pickling (to figure out how the objects get split up into component pieces)
and step through a real-world ConflictError situation.  Neither of which I
have time to do right now... sorry, Chris.

> > I bet it happens on a catalog write
> > for you.
>
> No, for once it has nothing to do with the catalog. ;-)
> This is a python product of ours that stores lots of data in somewhat
> nested, persistent dictionaries. How can this be improved so this
> doesn't happen?

Can you replicate the error on demand?  Or is it intermittent?  This should
be something covered in developer docs (along with a zillion other things).

I suppose a gross hack would be to raise the number of write retries on
conflicts in the ZODB code if you didn't want to analyze your app code.

But y'know, as I think more about it, I'm afraid I'm gonna need to bail on
this one by saying "I don't know" because it's one of those things where
learning exactly what is going on can chew up time like nobody's business,
because I'd need to step through the code line-by-line.  I shouldn't even be
in here.  :-(

Sorry,

C