[Zodb-checkins] SVN: ZODB/trunk/ Merge rev 37713 from 3.4 branch.

[Tim Peters]

Log message for revision 37714:
  Merge rev 37713 from 3.4 branch.
  
  Plug leaks in the ZEO client cache.
  
  ClientCache._evicted():  When deleting the last range of
  non-current tids for an oid, remove the list from the
  noncurrent dict instead of leaving an empty list sitting
  there forever.  This also required adjusting the side-
  effect dance in ClientCache._remove_noncurrent_revisions().
  
  FileCache._makeroom():  This was the major leak -- it
  neglected to remove an evicted object's Entry from the
  key2entry dict.
  

Changed:
  U   ZODB/trunk/NEWS.txt
  U   ZODB/trunk/src/ZEO/cache.py

-=-
Modified: ZODB/trunk/NEWS.txt
===================================================================
--- ZODB/trunk/NEWS.txt	2005-08-04 20:42:41 UTC (rev 37713)
+++ ZODB/trunk/NEWS.txt	2005-08-04 20:46:36 UTC (rev 37714)
@@ -29,6 +29,13 @@
   marked a savepoint as invalid after its first use.  The implementation has
   been repaired, to match the docs.
 
+ZEO client cache
+----------------
+
+- (3.5a6) Two memory leaks in the ZEO client cache were repaired, a
+  major one involving ``ZEO.cache.Entry`` objects, and a minor one involving
+  empty lists.
+
 Subtransactions are deprecated
 ------------------------------
 
@@ -166,6 +173,7 @@
 Following are dates of internal releases (to support ongoing Zope 2
 development) since ZODB 3.4's last public release:
 
+- 3.4.1b3 04-Aug-2005
 - 3.4.1b2 02-Aug-2005
 - 3.4.1b1 26-Jul-2005
 - 3.4.1a6 19-Jul-2005
@@ -192,6 +200,13 @@
   marked a savepoint as invalid after its first use.  The implementation has
   been repaired, to match the docs.
 
+ZEO client cache
+----------------
+
+- (3.4.1b3) Two memory leaks in the ZEO client cache were repaired, a
+  major one involving ``ZEO.cache.Entry`` objects, and a minor one involving
+  empty lists.
+
 Subtransactions
 ---------------
 

Modified: ZODB/trunk/src/ZEO/cache.py
===================================================================
--- ZODB/trunk/src/ZEO/cache.py	2005-08-04 20:42:41 UTC (rev 37713)
+++ ZODB/trunk/src/ZEO/cache.py	2005-08-04 20:46:36 UTC (rev 37714)
@@ -306,7 +306,10 @@
                 # 0x1E = invalidate (hit, discarding current or non-current)
                 self._trace(0x1E, oid, version, tid)
                 self.fc.remove((oid, old_tid))
-            del self.noncurrent[oid]
+            # fc.remove() calling back to _evicted() should have removed
+            # the list from noncurrent when the last non-current revision
+            # was removed.
+            assert oid not in self.noncurrent
 
     ##
     # If `tid` is None, or we have data for `oid` in a (non-empty) version,
@@ -433,7 +436,15 @@
             # we never expect the list to be very long.  So the
             # brute force approach should normally be fine.
             L = self.noncurrent[oid]
-            L.remove((o.start_tid, o.end_tid))
+            element = (o.start_tid, o.end_tid)
+            if len(L) == 1:
+                # We don't want to leave an empty list in the dict:  if
+                # the oid is never referenced again, it would consume RAM
+                # forever more for no purpose.
+                assert L[0] == element
+                del self.noncurrent[oid]
+            else:
+                L.remove(element)
 
     # If `path` isn't None (== we're using a persistent cache file), and
     # envar ZEO_CACHE_TRACE is set to a non-empty value, try to open
@@ -890,8 +901,8 @@
     # the end of the file, currentofs is reset to ZEC3_HEADER_SIZE first.
     # The number of bytes actually freed may be (and probably will be)
     # greater than nbytes, and is _makeroom's return value.  The file is not
-    # altered by _makeroom.  filemap is updated to reflect the
-    # evictions, and it's the caller's responsibilty both to fiddle
+    # altered by _makeroom.  filemap and key2entry are updated to reflect the
+    # evictions, and it's the caller's responsibility both to fiddle
     # the file, and to update filemap, to account for all the space
     # freed (starting at currentofs when _makeroom returns, and
     # spanning the number of bytes retured by _makeroom).
@@ -903,6 +914,7 @@
         while nbytes > 0:
             size, e = self.filemap.pop(ofs)
             if e is not None:
+                del self.key2entry[e.key]
                 self._evictobj(e, size)
             ofs += size
             nbytes -= size
@@ -912,7 +924,8 @@
     # Write Object obj, with data, to file starting at currentofs.
     # nfreebytes are already available for overwriting, and it's
     # guranteed that's enough.  obj.offset is changed to reflect the
-    # new data record position, and filemap is updated to match.
+    # new data record position, and filemap and key2entry are updated to
+    # match.
     def _writeobj(self, obj, nfreebytes):
         size = OBJECT_HEADER_SIZE + obj.size
         assert size <= nfreebytes