[ZODB-Dev] Partitioning a container

[Adrian van den Dries]

Here's my idea:

I've seen a couple of places with a storage pattern of "no deletes":
ie, when you've finished with an object, you "retire" it and it moves
into an archive.  This means your storage will grow indefinitely, so
it would be good to be able to partition your container into different
storages: current and archive, and tune either to the needs of the
application (temp storage for current, Oracle for archive, etc.)

Here's the catch: the object's path mustn't change: that is, the
container transparently decides the storage from which to pull the
object.  This would satisfy the content management ideal of
"persistent URLs".  For example:

  /news/1999/blah.html -> /news/_archive/1999/blah.html
  /news/2005/foo.html ->  /news/_current/2005/foo.html

Generalising further, even better would be the ability to define your
own partitioning strategy: the example that comes to mind is a large
user homepage farm, where you partition according to a member's
username (first letter, etc.), but the container still transparently
maps the object id.  There are probably many other uses.

I suspect this would potentially touch a lot of code with which I am
not entirely familiar, and I'm not sure where to start. (Mounts,
Acquisition, Traversable come to mind.) If it could be used with the
so-called standalone ZODB, that would make ZODB an *extremely*
powerful object database, but what is the status of mounting storages
in this context?  Also, an implementation at this level precludes the
possibility of using 30x redirects (which aren't the right solution
anyway).

Does this sound feasible?  Tickle anyone's itch?

I suspect AdaptableStorage is the place to experiment with this.

a.

-- 
 Adrian van den Dries                           adriand@flow.com.au
 Development team                               www.dev.flow.com.au
 FLOW Communications Pty. Ltd.                  www.flow.com.au