[ZODB-Dev] Notes on using wrapper storages for record transformation
Jim Fulton
jim at zope.com
Tue May 18 16:08:51 EDT 2010
The design of ZODB views pickles as opaque to storages. Storages
primarily store strings. I have always assumed that writing wrapper
storages for compression or encryption would be easy. I was wrong. :)
Storages sometimes need to know about pickles:
A. When doing garbage collection, a storage (or garbage collector)
needs to extract object references.
B. When doing conflict resolution, the conflict-resolution code needs
to be able to extract state from object records.
C. When doing blob-aware iteration, a storage needs to be able to
recognize blob records.
D. When raising conflict errors, the exception instances *want* to get
the object class name from the database record.
(A) was dealt with by the existing way the referencesf function was
passed into storage pack functions.
I dealt (B) and (C) by expanding and renaming the interface of objects
passed to the storage registerDB call. These objects now provide an
up-call mechanism from a storage to anything that might wrap it,
including databases, storage servers, and intermediate wrapping
layers.
I punted on (D). You can't always get what you want. :)
I'm mostly convinced what I've done is reasonable, although I'm just a
little uncomfortable. It has always been the theory that storages
can't count on the contents of database records. Now that is
reality.
I've decided I really want to provide a database compression option. I
could have built something directly into ZODB or used a wrapper
storage. I chose the later because it was more flexible. I think I
would have run into the same issues without a wrapper storage.
Soon I expect to release a wrapper storage implementation that
provides record compression.
Jim
--
Jim Fulton
More information about the ZODB-Dev
mailing list