[ZODB-Dev] Implementing Storage Decorators

[Jim Fulton]

I really like the model of providing storage functionality by  
combining storage decorators. A decorator is a component that  
augments and possibly changes the interfaces provided by a storage.  
For example, the BlobStorage adds support for BLOBS to existing  
storages, using a wrapped storage to hold object data.  Version 2 of  
Zope Replication Services (ZRS2) is implemented with storage  
decorators.  It would be useful to have decorators that provides  
services like statitics reporting, data compression, and data  
encryption.

A decorator will wrap some storage and delegate some calls to it.  In  
some cases, it will modify a method, providing it's own logic and  
possibly calling the method of the wrapped storage.  In other cases,  
it will use methods of the underlying storage without change.  I  
suspect that using underlying storage methods is most common.

A question arises as to how to implement storage decorators.   
BlobStorage uses zope.proxy.  This fully automates delegating to  
methods of underlying storages without change and makes overriding  
methods easy enough.  This is the approach that BlobStorage uses.

Another aproach is to copy storage methods into the decorator's  
instance dictionary during initialization of the decorator.  This is  
the approach taked by databases, connections, and storage servers.   
This is also the approach taken by ZRS2.

When BlobStorage was implemented, the ZODB storage interfaces were  
almost undefined. It was very difficult to know which methods were  
available and which needed to be proxied.  Using zope.proxy was the  
sanest approach.

Now that I think we have well-defined storage interfaces, the method  
copying approach is more appealing to me for two reasons:

- It is explicit.  The author of the decorator has to deal with every  
method, even if only by naming the methods they want to copy.

   I like being explicit, especially for something as complex as the  
storage interface.  BaseStorage is an example of why I don't like  
implicit. It makes it easy to implicitly implement methods incorrectly.

- If we used it everywhere, we could avoid a dependency on zope.proxy.

Some downsides of copying:

- It's a pain to support optional interfaces.  The copying code has  
to do some duck typing to decide what to copy.  This might be cleaned  
up if interface declarations were used.  We haven't bothered to make  
interface declarations for storages to date, because we haven't had  
decent interfaces to declare.

- It's a pain to support old ZODB versions.  Hopefully, going  
forward, this won't matter, but who knows.

- If we want to start using interface declarations, the decorators  
will have to copy interface declations from the wrapped storage. I  
think that this can be as simple as::

     zope.interface.directlyProvides(
        self, zope.interface.providedBy(self._storage))

   There's a base class that automates decorator interface- 
declaration management when using zope.proxy.

I think I still rather like explicit, but I'm on the fence about  
which approach is best.  What do other people think?

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org