[ZODB-Dev] [Persistent] STICKY mechanism unsafe

[Dieter Maurer]

Jim Fulton wrote at 2007-8-20 10:45 -0400:
> ...
>Dieter appears to have been bitten by this and he is one of we. :)
>
>We, and I presume he, can be bitten by a Python function called from  
>BTree code calling back into the code on the same object.  This is  
>possible, for example, in a __cmp__ or related method.  I assume that  
>this is what happened to Dieter.  Obviously, this would be a fairly  
>"special" comparison method.

I am not yet sure what really has bitten us -- I am not even sure
whether the object was really deactivated or some memory corruption
caused the object's tail to be overwritten by "0".

When the SIGSEGV had hit, usually a bucket in a "TextIndexNG3.lexicon"
was affected. This lexicon uses "BTrees" in a very innocent way.
Its keys are integers and strings -- no fancy "__cmp__" method is
involved.

Moreover, we need two things for the deactivation to happen:
the STICKY mechanism must fail *AND* a deactivation must be
called for.

In our Zope/ZODB version, deactivation is done only at transaction
boundaries (it is an early ZODB 3.4 version where snapshops did not
yet call "incrgc"). Therefore, some "commit" would need to be
done during the "BUCKET_SEARCH" call.

The only conceivable cause appears to me that a different thread
modified the bucket and called "abort". This would mean
a persistency bug (concurrent use of a persistent object by
several threads). I tried to find such a bug in "TextIndexNG3", but
failed.


The problem appears only very rarely -- about 1 to 2 times in about 1 to 2
month. When I analysed the problem in the past, I failed to look
at the object's persistent state (it would have told me whether
the object has been deactivated or overwritten). I just noticed
that the object's head was apparently intact while the object's true
data was 0. Only a few days ago, I recognized that this could
have been the effect of a deactivation.



-- 
Dieter