[Zodb-checkins] CVS: ZODB4/src/zodb/storage - base.py:1.13.4.1

[Barry Warsaw]

Update of /cvs-repository/ZODB4/src/zodb/storage
In directory cvs.zope.org:/tmp/cvs-serv1326/src/zodb/storage

Modified Files:
      Tag: opaque-pickles-branch
	base.py 
Log Message:
The start of opaque pickles (from the p.o.v. of the storages).  This
will eventually allow us to pass compressed pickles to the storage if
we want.

The approach basically changes store() so that the data argument is a
2-tuple of the pickle and the list of oids referenced in the pickle.
This is the first step in the changes, but currently, only Berkeley
storages natively store the refs included in the store() API call.

Changes here include:

- BerkeleyBase.__init__(): Create the 'referents' table to hold the
  oid->refs mapping.

- _update(), _splitoids(): Rewrite _update() to take the list of
  referents as a string of 8-byte oids, which is split into oids via
  _splitoids().

- Use ZERO from zodb.interfaces


=== ZODB4/src/zodb/storage/base.py 1.13 => 1.13.4.1 ===
--- ZODB4/src/zodb/storage/base.py:1.13	Wed Feb  5 18:28:32 2003
+++ ZODB4/src/zodb/storage/base.py	Mon Feb 10 17:58:52 2003
@@ -41,14 +41,12 @@
     berkeley_is_available = False
 
 from zodb.timestamp import newTimeStamp, TimeStamp
-from zodb.interfaces import ITransactionAttrs
+from zodb.interfaces import ITransactionAttrs, ZERO
 from zodb.storage.interfaces import StorageTransactionError, ReadOnlyError
 # BaseStorage provides primitives for lock acquisition and release, and a host
 # of other methods, some of which are overridden here, some of which are not.
 from zodb.lockfile import lock_file
-from zodb.serialize import findrefs
 
-ZERO = '\0'*8
 GBYTES = 1024 * 1024 * 1000
 JOIN_TIME = 10
 
@@ -500,6 +498,7 @@
         self._serials = self._setupDB('serials', db.DB_DUP)
         self._pickles = self._setupDB('pickles')
         self._refcounts = self._setupDB('refcounts')
+        self._referents = self._setupDB('referents')
         self._oids = self._setupDB('oids')
         self._pending = self._setupDB('pending')
         self._packmark = self._setupDB('packmark')
@@ -700,8 +699,13 @@
         os.unlink(lockfile)
 
     # A couple of convenience methods
-    def _update(self, deltas, data, incdec):
-        for oid in findrefs(data):
+    def _splitoids(self, referents):
+        num, extra = divmod(len(referents), 8)
+        assert extra == 0, referents
+        return struct.unpack('>' + ('8s' * num), referents)
+
+    def _update(self, deltas, referents, incdec):
+        for oid in self._splitoids(referents):
             rc = deltas.get(oid, 0) + incdec
             if rc == 0:
                 # Save space in the dict by zapping zeroes