[Zodb-checkins] SVN: ZODB/trunk/ Merge rev 38051 from 3.4 branch.

[Tim Peters]

Log message for revision 38054:
  Merge rev 38051 from 3.4 branch.
  
  Code and new test to ensure that making a savepoint triggers cache gc.
  

Changed:
  U   ZODB/trunk/NEWS.txt
  U   ZODB/trunk/src/ZODB/Connection.py
  U   ZODB/trunk/src/ZODB/tests/testConnectionSavepoint.py

-=-
Modified: ZODB/trunk/NEWS.txt
===================================================================
--- ZODB/trunk/NEWS.txt	2005-08-23 21:45:13 UTC (rev 38053)
+++ ZODB/trunk/NEWS.txt	2005-08-23 22:01:14 UTC (rev 38054)
@@ -16,6 +16,13 @@
 Savepoints
 ----------
 
+- (3.5a8) As for deprecated subtransaction commits, the intent was
+  that making a savepoint would invoke incremental garbage collection on
+  Connection memory caches, to try to reduce the number of objects in
+  cache to the configured cache size.  Due to an oversight, this didn't
+  happen, and stopped happening for subtransaction commits too.  Making a
+  savepoint (or doing a subtransaction commit) does invoke cache gc now.
+
 - (3.5a3) When a savepoint is made, the states of objects modified so far
   are saved to a temporary storage (an instance of class ``TmpStore``,
   although that's an internal implementation detail).  That storage needs
@@ -62,12 +69,16 @@
 
   to::
 
-      transaction.savepoint()
+      transaction.savepoint(True)
 
-  That is, make a savepoint, and forget it.  In rarer cases, a
-  subtransaction commit is followed later by a subtransaction abort.  In
-  that case, change the initial::
+  That is, make a savepoint, and forget it.  As shown, it's best to pass
+  ``True`` for the optional ``optimistic`` argument in this case:  because
+  there's no possibility of asking for a rollback later, there's no need
+  to insist that all data managers support rollback.
 
+  In rarer cases, a subtransaction commit is followed later by a
+  subtransaction abort.  In that case, change the initial::
+
       transaction.commit(1)
 
   to::
@@ -188,6 +199,26 @@
   in ZODB's C code were rewritten in minor ways.
 
 
+What's new in ZODB3 3.4.2a1?
+============================
+Release date: DD-MMM-2005
+
+Following are dates of internal releases (to support ongoing Zope 2
+development) since ZODB 3.4's last public release:
+
+- 3.4.2a1 DD-MMM-2005
+
+Savepoints
+----------
+
+- (3.4.2a1) As for deprecated subtransaction commits, the intent was
+  that making a savepoint would invoke incremental garbage collection on
+  Connection memory caches, to try to reduce the number of objects in
+  cache to the configured cache size.  Due to an oversight, this didn't
+  happen, and stopped happening for subtransaction commits too.  Making a
+  savepoint (or doing a subtransaction commit) does invoke cache gc now.
+
+
 What's new in ZODB3 3.4.1?
 ==========================
 Release date: 09-Aug-2005
@@ -250,12 +281,16 @@
 
   to::
 
-      transaction.savepoint()
+      transaction.savepoint(True)
 
-  That is, make a savepoint, and forget it.  In rarer cases, a
-  subtransaction commit is followed later by a subtransaction abort.  In
-  that case, change the initial::
+  That is, make a savepoint, and forget it.  As shown, it's best to pass
+  ``True`` for the optional ``optimistic`` argument in this case:  because
+  there's no possibility of asking for a rollback later, there's no need
+  to insist that all data managers support rollback.
 
+  In rarer cases, a subtransaction commit is followed later by a
+  subtransaction abort.  In that case, change the initial::
+
       transaction.commit(1)
 
   to::

Modified: ZODB/trunk/src/ZODB/Connection.py
===================================================================
--- ZODB/trunk/src/ZODB/Connection.py	2005-08-23 21:45:13 UTC (rev 38053)
+++ ZODB/trunk/src/ZODB/Connection.py	2005-08-23 22:01:14 UTC (rev 38054)
@@ -1063,7 +1063,13 @@
         self._registered_objects = []
 
         state = self._storage.position, self._storage.index.copy()
-        return Savepoint(self, state)
+        result = Savepoint(self, state)
+        # While the interface doesn't guarantee this, savepoints are
+        # sometimes used just to "break up" very long transactions, and as
+        # a pragmatic matter this is a good time to reduce the cache
+        # memory burden.
+        self.cacheGC()
+        return result
 
     def _rollback(self, state):
         self._abort()

Modified: ZODB/trunk/src/ZODB/tests/testConnectionSavepoint.py
===================================================================
--- ZODB/trunk/src/ZODB/tests/testConnectionSavepoint.py	2005-08-23 21:45:13 UTC (rev 38053)
+++ ZODB/trunk/src/ZODB/tests/testConnectionSavepoint.py	2005-08-23 22:01:14 UTC (rev 38054)
@@ -85,8 +85,63 @@
 
     >>> db.close()
     """
-    
 
+def testSavepointDoesCacheGC():
+    """\
+Although the interface doesn't guarantee this internal detail, making a
+savepoint should do incremental gc on connection memory caches.  Indeed,
+one traditional use for savepoints (started by the older, related
+"subtransaction commit" idea) is simply to free memory space midstream
+during a long transaction.  Before ZODB 3.4.2, making a savepoint failed
+to trigger cache gc, and this test verifies that it now does.
+
+    >>> import ZODB
+    >>> from ZODB.tests.MinPO import MinPO
+    >>> from ZODB.MappingStorage import MappingStorage
+    >>> import transaction
+    >>> CACHESIZE = 5  # something tiny
+    >>> LOOPCOUNT = CACHESIZE * 10
+    >>> st = MappingStorage("Test")
+    >>> db = ZODB.DB(st, cache_size=CACHESIZE)
+    >>> cn = db.open()
+    >>> rt = cn.root()
+
+Now attach substantially more than CACHESIZE persistent objects to the root:
+
+    >>> for i in range(LOOPCOUNT):
+    ...     rt[i] = MinPO(i)
+    >>> transaction.commit()
+
+Now modify all of them; the cache should contain LOOPCOUNT MinPO objects
+then, + 1 for the root object:
+
+    >>> for i in range(LOOPCOUNT):
+    ...     obj = rt[i]
+    ...     obj.value = -i
+    >>> len(cn._cache) == LOOPCOUNT + 1
+    True
+
+Making a savepoint at this time used to leave the cache holding the same
+number of objects.  Make sure the cache shrinks now instead.
+
+    >>> dummy = transaction.savepoint()
+    >>> len(cn._cache) <= CACHESIZE + 1
+    True
+
+Verify all the values are as expected:
+
+    >>> failures = []
+    >>> for i in range(LOOPCOUNT):
+    ...     obj = rt[i]
+    ...     if obj.value != -i:
+    ...         failures.append(obj)
+    >>> failures
+    []
+
+    >>> transaction.abort()
+    >>> db.close()
+"""
+
 def test_suite():
     return unittest.TestSuite((
         doctest.DocFileSuite('testConnectionSavepoint.txt'),