[ZODB-Dev] Tracking down a freeze (deadlock?)

[Tim Peters]

[Tim]
>> ...
>> Or are "the rules of persistence" documented in some place other than
>> [the ZODB/ZEO Programming Guide] (if so, where?)?

[Jim]
> I have no idea.

In that case, updating the rules of persistence is very easy -- there's
nothing to be changed <wink>.

Googling on "rules of persistence" (including the quotes!) turns up lots of
hits, all seeming to originate with your ZODB presentation in 2000, which
has a section title with that name:

http://www.python.org/workshops/2000-01/proceedings/papers/fulton/zodb3.html

AFAIK, that wasn't updated, and the source for it wasn't checked in.  Right?
IOW, the only user-level doc I know of for ZODB that ever gets updated is
the ZODB/ZEO Programming Guide (which is checked in to the project, and gets
minimal attention from time to time).

> ...
> In earlier messages, It looked like someone tracked down the deadlock to
> a __del__ causing an object to get loaded when an object was invalidated.

Florent showed a traceback of that form, yes.

> (An object was invalidated, which caused another object's refcount to
> fall to 0, which caused it's __del__ to be called.  The __del__ caused
> some object to be reloaded, which called setstate on the connection.  The
> setstate tried to get the invalidation lock, which was already held by
> the code doing the invalidation.)

That's a nastier problem than we've been discussing here, right?  Meaning
that there's nothing special about a __del__ method attached to a
*persistent* class in this scenario -- any object (persistent or not) with a
__del__ method could trigger a deadlock ... like so:

"""
import ZODB
from ZODB.FileStorage import FileStorage
from Persistence import Persistent

st = FileStorage('temp.fs')
db = ZODB.DB(st)
cn = db.open()
rt = cn.root()

class C(Persistent):
    def __init__(self, attr=None):
        self.attr = attr

class D:
    def __del__(self):
        print 'in __del__'
        print cn.root()['c2'].attr

rt['c1'] = C(D())
rt['c2'] = C("success")
get_transaction().commit()

cn.invalidate({rt['c1']._p_oid: 1, rt['c2']._p_oid: 1})
print "flushing"
cn._flush_invalidations()
print "done flushing"
"""

That prints "in __del__" as a side effect of calling _flush_invalidations(),
then deadlocks on the next line.  Hmm!  Can't account for this:  sometimes
it *doesn't* deadlock, completing with "success".  Yikes.

OK, that depends on the hash codes of the oids <heh>.  Reliable deadlock:

"""
import ZODB
from ZODB.FileStorage import FileStorage
from Persistence import Persistent

st = FileStorage('temp.fs')
db = ZODB.DB(st)
cn = db.open()
rt = cn.root()

class C(Persistent):
    def __init__(self, attr=None):
        self.attr = attr

class D:
    def __del__(self):
        print 'in __del__'
        print cn.root()['c2'].attr

rt['c1'] = C(D())
rt['c2'] = C("success")
get_transaction().commit()

cn.invalidate({rt['c2']._p_oid: 1})
print "flushing 1"
cn._flush_invalidations()
cn.invalidate({rt['c1']._p_oid: 1})
print "flushing 2"
cn._flush_invalidations()
print "done flushing"
"""

> This is very similar to the problem I ran into with ZClasses. When a
> ZClass is invalidated, we want to reload the state.  Originally, this
> lead to a deadlock.  I had to change some the invalidation strategy so
> that the code the code that performes invalidations doesn't hold the
> invalidation lock.

I'd much rather spend time on that than trying to defang hostile __del__
methods.