[Zope] Trying to trap ConflictError

[Jonathan]

----- Original Message ----- 
From: "Dennis Allison" <allison at shasta.stanford.edu>
To: "Jonathan" <dev101 at magma.ca>
Cc: <zope at zope.org>
Sent: Tuesday, July 04, 2006 1:27 PM
Subject: Re: [Zope] Trying to trap ConflictError


>
> Johnathan,
>
> I have been lurking, following your thread, but not paying full
> attention.  Excuse me if thes points have been made before.
>
> There are many long threads on Conflict Errors.  You should read them.
> Note that there have been significant changes in management strategy
> beginning with Zope 2.9.X.
>
> A "Conflict Error" is not really an error in the usual sense;  it is an
> indication of conflicting transactions in the ZODB, that is, transactions
> have violated the optimistic locking used by the ZODB.  In general,
> Conflict Errors are resolved without problem.  The resolution usually
> involves rolling back the older transaction and committing the newer
> transaction.  When the actions associated with the rolled back transaction
> are not themselves fully transactional (for example, a file write) there
> can be unanticipated side effects.
>
> Your conflicts appear to be in Temporary Storage which is used for
> Session Variables.  The version that ships with Zope is flexible and
> complex and, IMHO, overkill.  Moreover, it has some unexpected properties,
> the most significant is that, even though session variables are stored in
> RAM, it occasionally uses the ZODB to manage session variable expiration.
> You can often decrease the number of session variable induced conflicts by
> increasing the lifetime of session variables.
>

As both you and Andreas have recommended, I have started researching 'ZODB 
Conflict Resolution' and if I come up with generic/reusable external method 
application-level solution I will post it.

By way of providing more information:

I am not using sessions.  I am using  TemporaryStorage to hold small, 
custom-made, short-lived image objects (the use case requires that each user 
views a static image with a custom overlay, and that the user accesses this 
customized image via an html img tag). The custom images are viewed only 
once, and then can be deleted. Hence the use of TemporaryStorage (to 
eliminate many disk writes and ZODB bloating). The image objects stored in 
TemporaryStorage are never overwritten, edited etc (which is causing me some 
confusion as to why the conflict errors are occurring), they are simply 
created, stored in the TemporaryStorage folder (actually a BTreeFolder2 
folder contained within the TemporaryStorage folder) and then 2 minutes 
later are deleted.  I am guessing that the conflicts are occuring as the 
TemporaryStorage is trying to delete old images while new image object 'add 
requests' are piling up, but I have yet to determine the cause of the 
conflict with 100% certainty (conflicts only occur under heavy loading, and 
not always in the same place in the load testing).


Thanks again for everyone's help!

Jonathan