[ZODB-Dev] Tracking down a freeze (deadlock?)

[Tim Peters]

[Dieter Maurer]
> I do not think that "__del__" (at least with its current semantics) is
> useful for persistent classes.
>
> The only use case I can think of is to clean up non-persistent resources
> associated with the instance. However, the corresponding attributes (they
> need to be volatile) are usually already lost when the "__del__" is
> called (by a preceeding invalidation).

If it's desired to clean up a resource whenever a volatile attribute goes
away, then a clean way to do that is to bind the volatile attribute to an
instance of a simple non-persistent class where the *latter* has a __del__
method, and where the latter class's only purpose in life is to clean up a
resource via its __del__.  That's the best way to do
resource-cleanup-via-__del__ in the absence of persistence too (then, for a
start, instances of "the real" class don't have a __del__ method at all, so
are immune to getting in trouble with cyclic gc).

> However, deadlocking (or looping indefinitely) is not the right way to
> flag an unuseful feature (such as "__del__" in a persistent class).
> It is far too difficult to track down the cause of the
> deadlocking/spinning.

I have no reason to suspect that the possible interactions between
persistence and __del__ were considered during ZODB's design.  If so, that
makes "flag" too strong a phrasing.  It seems more accidents due to using
the system in unanticipated and dubious ways.  I don't think "dubious" is
too strong a phrasing, since the code in question immediately looked fishy
to Florent.

We've had one report of this particular accident across all recorded history
that I know of, and fixing bugs involves a series of tradeoffs too.  Since
there isn't even a Collector issue on this one, it's got no priority
compared to the 40-or-so ZODB-related Collector issues already open.  If
there were a Collector issue for this one(*), I don't think it would get
high priority anyway, because of the historical rarity of the problem; the
ease of avoiding it to begin with ("so don't do that" at a syntactic level);
and the likelihood that if ZODB were to start refusing to accept persistent
classes with __del__ methods, *somebody's* code would stop working.  For
example, the code of whoever it was who bumped into the infinite loop in the
Connection cache by mixing persistence with __del__ with the LRU cache.
Maybe even other of Florent's code:  he didn't say he thought it was dubious
for a persistent object to *have* a __del__ method, just that it was dubious
for the __del__ method to try to do ZODB operations.

I'm certainly not opposed to "doing something" about this one.  The reality
is I can't make time for it without dropping something else, though, and as
I judge the tradeoffs it's not more important than anything else.  If it's
important enough to someone else to do the work, cool!


(*) OK, there is now:

     http://www.zope.org/Collectors/Zope/1718

As you can guess from the above, it's at the bottom of my list.