[Zodb-checkins] CVS: ZODB3/bsddb3Storage/bsddb3Storage - Minimal.py:1.12.4.1

[Barry Warsaw]

Update of /cvs-repository/ZODB3/bsddb3Storage/bsddb3Storage
In directory cvs.zope.org:/tmp/cvs-serv14701/bsddb3Storage/bsddb3Storage

Modified Files:
      Tag: bdb-nolocks
	Minimal.py 
Log Message:
Checking this into the bdb-nolocks branch until I can rewrite Full
and/or fix the current failures.  This branch utilizes some recent
insights into building BDB based storages to hopefully produce
storages that don't run out of locks and perform close to FileStorage
(depending on various optimizing configurations).  Key insight: don't
use BDB lock subsystem, but instead rely on the application locks
we're already using around updates.  This way we can't run out of BDB
locks no matter how many pages we'll touch.

[I also forgot to mention in the previous commit to BerkeleyBase.py
that we're also optimistically writing data so we can eliminate the
need for an intermediate log file.]

Changes here include:

- Complete rewrite of the Minimal storage.  See the comments in the
  file for the data model.  Life is so much simpler without the
  CommitLog, or lock exhaustion worries.

  One big change is that the storage will do autorecovery in cases
  where a crash occurred between the vote and the end of the finish.
  Another big change is that we're using reference counting so we
  never need to pack.  Yippee!

  XXX still need whitebox tests for refcount correctness and for
  auto-recovery.

- whitespace normalization.


=== ZODB3/bsddb3Storage/bsddb3Storage/Minimal.py 1.12 => 1.12.4.1 === (406/506 lines abridged)
--- ZODB3/bsddb3Storage/bsddb3Storage/Minimal.py:1.12	Tue Feb 12 17:33:09 2002
+++ ZODB3/bsddb3Storage/bsddb3Storage/Minimal.py	Mon Sep  9 18:52:51 2002
@@ -2,20 +2,17 @@
 #
 # Copyright (c) 2001, 2002 Zope Corporation and Contributors.
 # All Rights Reserved.
-# 
+#
 # This software is subject to the provisions of the Zope Public License,
 # Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
 # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
 # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
 # FOR A PARTICULAR PURPOSE
-# 
+#
 ##############################################################################
 
 """Berkeley storage without undo or versioning.
-
-See Full.py for an implementation of Berkeley storage that does support undo
-and versioning.
 """
 
 __version__ = '$Revision$'[-2:][0]
@@ -25,224 +22,301 @@
 # PyBSDDB3.
 from bsddb3 import db
 
-# BerkeleyBase.BerkeleyBase class provides some common functionality for both
-# the Full and Minimal implementations.  It in turn inherits from
-# ZODB.BaseStorage.BaseStorage which itself provides some common storage
-# functionality.
+# BerkeleyBase class provides some common functionality for BerkeleyDB-based
+# storages.  It in turn inherits from BaseStorage which itself provides some
+# common storage functionality.
 from BerkeleyBase import BerkeleyBase
-from CommitLog import PacklessLog
 from ZODB import POSException
-from ZODB import utils
+from ZODB.utils import U64, p64
+from ZODB.referencesf import referencesf
+
+ABORT = 'A'
+COMMIT = 'C'
+PRESENT = 'X'
+ZERO = '\0'*8
+
 
 

[-=- -=- -=- 406 lines omitted -=- -=- -=-]

+
+    def load(self, oid, version):
+        if version <> '':
+            raise POSException.Unsupported, 'versions are not supported'
         self._lock_acquire()
         try:
-            # Build an index only of those objects reachable from the root.
-            # Unfortunately, we do this in memory, so the memory footprint of
-            # packing may still be substantial.
-            #
-            # Known root objects are kept in this list and as new ones are
-            # found, their oids are pushed onto the front of the list. It is
-            # also added to the seen dictionary, which keeps track of objects
-            # we've seen already.  When roots is empty, we're done visiting
-            # all the objects.
-            roots = ['\0\0\0\0\0\0\0\0']
-            seen = {}
-            while roots:
-                # Get the next oid from the roots list
-                oid = roots.pop()
-                # Skip it if we've already seen it
-                if seen.has_key(oid):
-                    continue
-                # Get the pickle corresponding to the object id and scan it
-                # for references to other objects.  This is done by the
-                # magical `getrefsfunc' function given as an argument.
-                pickle = self._pickles[oid]
-                seen[oid] = 1
-                # This will prepend any new oids we'll need to scan
-                getrefsfunc(pickle, roots)
-            # Now, go through every oid for which we have a pickle, and if we
-            # have not seen it, then it must be garbage (because it was never
-            # reached from one of the roots).  In that case, delete its entry
-            # in the pickle index.
-            for oid in self._pickles.keys():
-                if not seen.has_key(oid):
-                    del self._pickles[oid]
+            # Get the current serial number for this object
+            serial = self._getCurrentSerial(oid)
+            if serial is None:
+                raise KeyError, 'Object does not exist: %r' % oid
+            # Get this revision's pickle data
+            return self._pickles[oid+serial], serial
         finally:
             self._lock_release()
+
+    def modifiedInVersion(self, oid):
+        # So BaseStorage.getSerial just works.  Note that this storage doesn't
+        # support versions.
+        return ''