[Zope-dev] AdaptableStorage status update

[Shane Hathaway]

I've been holding off status updates on the AdaptableStorage project 
until I could say I'm confident that the approach will work and it's 
easy enough to write your own serialization components.  After numerous 
refactorings, I'm pleased to say that it actually works and does 
everything I hoped!

AdaptableStorage lets you store ZODB objects in your own database and in 
your format, *without* using special content classes.  You can use a 
filesystem directory, a relational database, or whatever kind of 
database you might have as a ZODB.

This is very different from all previous solutions.  LocalFS, ExtFile, 
CMF directory views, and other similar products require you to use 
special content classes that store themselves outside the ZODB. 
OracleStorage and CVSFile store all objects as pickles.  Other solutions 
have similar issues.  AdaptableStorage, however, reuses as much of ZODB 
as possible to achieve orthogonal persistence to arbitrary databases and 
formats.

This is only a status report, not an announcement that it's finished, so 
the documentation is not done yet.  But there are interfaces, unit 
tests, and a basic diagram.  You can use DBTab to mount an 
AdaptableStorage instance.  As a test, I have successfully used 
AdaptableStorage to create a CMF site on both the filesystem and in a 
Postgres database.

Here's a way to get a preview.  (It may not work in MS Windows yet.) 
Get a very recent version of Zope 2.6 (or perhaps even the HEAD).  Then 
check out AdaptableStorage and DBTab from CVS:

cvs -d :pserver:anonymous@cvs.zope.org:/cvs-repository co 
Products/AdaptableStorage
cvs -d :pserver:anonymous@cvs.zope.org:/cvs-repository co Products/DBTab

Move the two products to the Products directory of your Zope instance. 
Create a dbtab.conf file in your <Zope> directory with the following:



[Storage: FS]
type=Zope2FSStorage
basepath=%(CLIENT_HOME)s/myfs
volatile=1

[Database: FS]
class=Zope2FSDatabase
mount_paths=/fs
auto_create=1



Don't forget to create the directory "myfs" like this:

mkdir var/myfs

Start Zope and choose "Add DBTab Mount Point" in the management UI.  You 
should get a checkbox next to the "/fs" mount.  Create that mount point. 
  Now visit /fs and add stuff.  Watch var/myfs/fs, which will contain 
all the objects you create.

You can't yet copy/paste, import, or export, but you can add just about 
any kind of object.  The files under var/myfs/fs currently contain 
pickles, but that's only because AdaptableStorage doesn't have a mapper 
registered for most kinds of objects and it falls back to the default 
mappers.  But AdaptableStorage does know about Folders and it creates a 
directory for them.  It also works the other way around: if you create a 
directory on the filesystem, you'll see it in Zope as a Folder.

If you're adventurous, you can also set up a SQL connection using 
psycopg and the following dbtab.conf entries:


[Storage: Postgres]
type=Zope2SQLStorage
params=
table_prefix=zodb
volatile=1

[Database: Postgres]
class=Zope2SQLDatabase
mount_paths=/sql
auto_create=1


You might observe that it's a little less responsive than regular ZODB. 
   Edit dbtab.conf, changing "volatile=1" to "volatile=0", to get the 
full benefits of the ZODB cache.  The downside is that changes you make 
on the filesystem won't be reflected immediately, but in a lot of 
environments, that is acceptable.

The AdaptableStorage I presented at the O'Reilly Open Source conference 
was specific to Zope 2.  This version is not.  In fact, most of it isn't 
even specific to ZODB.  Its core is a general serialization framework 
with pattern names drawn from Martin Fowler's book.

Let me know if you run into troubles.  I've been testing it with the 
Zope HEAD on Python 2.2.2.  This is the product of many months of 
inspiration and perspiration. :-)  It's not perfect, but the basic 
concepts and interfaces are in good shape.  Comments welcome.

Shane