[Zope3-checkins] CVS: ZODB3/ZODB/tests - testCache.py:1.13.10.5

[Tim Peters]

Update of /cvs-repository/ZODB3/ZODB/tests
In directory cvs.zope.org:/tmp/cvs-serv22870/ZODB/tests

Modified Files:
      Tag: Zope-2_7-branch
	testCache.py 
Log Message:
Collector #1208:  Infinite loop in cPickleCache.

This fixes it, based on an approach suggested by Toby Dickenson.
The triggering condition wasn't entirely sane, so was very rare:
a persistent object with a __del__ method that referenced an
attribute of self.

scan_gc_items():  Look at objects accessed by __setattr__ and
__del__ calls no more than once.

New test checkMinimizeTerminates():  This spawns a thread that
will in fact run forever if you don't recompile cPickleCache.c.
The test suite will keep running, but checkMinimizeTerminates
will fail, and all future calls to cacheMinimize() will be
effectively ignored (so other bad things may happen as a
result).


=== ZODB3/ZODB/tests/testCache.py 1.13.10.4 => 1.13.10.5 ===
--- ZODB3/ZODB/tests/testCache.py:1.13.10.4	Wed May 19 13:24:57 2004
+++ ZODB3/ZODB/tests/testCache.py	Wed May 19 16:24:37 2004
@@ -22,6 +22,7 @@
 import time
 import types
 import unittest
+import threading
 
 import ZODB
 import ZODB.MappingStorage
@@ -105,6 +106,60 @@
         self.db.cacheMinimize(0)
         new_size = self.db.cacheSize()
         self.assert_(new_size < old_size, "%s < %s" % (old_size, new_size))
+
+    def checkMinimizeTerminates(self):
+        # This is tricky.  cPickleCache had a case where it could get into
+        # an infinite loop, but we don't want the test suite to hang
+        # if this bug reappears.  So this test spawns a thread to run the
+        # dangerous operation, and the main thread complains if the worker
+        # thread hasn't finished in 30 seconds (arbitrary, but way more
+        # than enough).  In that case, the worker thread will continue
+        # running forever (until killed externally), but at least the
+        # test suite will move on.
+        #
+        # The bug was triggered by having a persistent object whose __del__
+        # method references an attribute of the object.  An attempt to
+        # ghostify such an object will clear the attribute, and if the
+        # cache also releases the last Python reference to the object then
+        # (due to ghostifying it), the __del__ method gets invoked.
+        # Referencing the attribute loads the object again, and also
+        # puts it back into the cPickleCache.  If the cache implementation
+        # isn't looking out for this, it can get into an infinite loop
+        # then, endlessly trying to ghostify an object that in turn keeps
+        # unghostifying itself again.
+        class Worker(threading.Thread):
+
+            class CantGetRidOfMe(MinPO):
+                def __init__(self, value):
+                    MinPO.__init__(self, value)
+                    self.an_attribute = 42
+
+                def __del__(self):
+                    # Referencing an attribute of self causes self to be
+                    # loaded into the cache again, which also resurrects
+                    # self.
+                    self.an_attribute
+
+            def __init__(self, testcase):
+                threading.Thread.__init__(self)
+                self.testcase = testcase
+
+            def run(self):
+                conn = self.testcase.conns[0]
+                r = conn.root()
+                d = r[1]
+                for i in range(len(d)):
+                    d[i] = self.CantGetRidOfMe(i)
+                get_transaction().commit()
+
+                self.testcase.db.cacheMinimize(0)
+
+        w = Worker(self)
+        w.start()
+        w.join(30)
+        if w.isAlive():
+            self.fail("cacheMinimize still running after 30 seconds -- "
+                      "almost certainly in an infinite loop")
 
     # XXX don't have an explicit test for incrgc, because the
     # connection and database call it internally