[Zope-dev] Re: [ZCM] [ZC] 869/ 5 Comment "Broken transaction handling in case of exceptions"

[Toby Dickenson]

On Monday 07 April 2003 5:19 pm, Florent Guillaume wrote:

> > > Caches routinely pass persistent objects from one transaction to the
> > > next.
> >
> > Are there any examples in stock Zope 2? I would consider this a bug.

Thanks for the pointers. All are in code that I am not familiar with, so I am 
not entirely confident in these analysis:

> ZCTextIndex caches a persistent lexicon in _v_lexicon.

This class does have a clear() method, but I cant see where it is called from. 
I am assuming it is never called.

At an application level this cache is not transparent. I think the following 
sequence is possible:
1. One transaction calls getLexicon. The appropriate object is found by name 
through acquisition, and placed in the cache.
2. Another transaction replaces the lexicon object with a different one.
3. A third transaction calls getLexicon again and gets the cached object, not 
the new object. This can last indefinitely, until Zope is shut down. 

At a zodb level:
4. Another transaction calls getLexicon and the return value is assigned as an 
attribute of a persistent object. The transaction commits. Things might 
continue to work until that lexicon object is deactivated (that is, possibly 
not until Zope is shut down). After that, there is no guarantee that the 
lexicon object can be accessed. Attempted accesses may give a POSKeyError. If 
this happens you will need to restore a backup.

This should be 100% reproducable on bsddb.Minimal with no further steps. It 
would have purged the pickled state of the lexicon at the end of step 2 when 
the original lexicon object became unreachable from the zodb root. 
FileStorage will see this problem only if it is packed between step 3 and 4.

I think the fix here is to empty the cache at the end of each transaction that 
fills it. This fixes the application-level and zodb-level problems.  I would 
appreciate a comment from someone more familiar with this code.

In this scenario DirectoryStorage would raise an exeception in step, 
preventing the commit that corrupts the storage.

> DC.ZRDB.DA caches Bucket() which I think are persistent in _v_cache.

These objects are "persistent" in the sense that they derive from the 
persistent base class, but not in the sense that they are registered with a 
DB and written to disk. Only this second class of persistent objects cause 
problems when passed between transactions, so I believe this is safe at ZODB 
level.

At application level, I understand the purpose of this cache is to be 
non-transparent. There is code in place to ensure that the cache is regularly 
refreshed.

> CMFCore.MemberDataTool does a cache of persistent MemberData objects in
> _v_temps.

I dont have a CMF handy - I may check this tomorrow.

> Is all this wrong ?

?

-- 
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson