[Zope-dev] ZODB DA?

[Jim Fulton]

Bill Anderson wrote:
> 
> OK, this has been rolling around in my head for a while, thought I'd get
> it out and take it for a walk.
> 
> For many needs, the ZODB fits much better than an RDBMS.
> Unfortunately, there are some issues.
> 
> Linux x86 32 Bit VFS:
> Yes the 2sgb problem. Granted it can be fixed, but I believe there can
> be larger issues.
> 
> Just Plain Size:
> A big Data.fs can be a pain to backup, or to load. You may not
> neccesarily want to staore everything in your 'main'  ZODB. Maybe being
> able to use a seperate db for seperate uses would be handy, eg. an
> inventory that gives you ZODB benefits, but doesn't make your server
> 'forever' to initialize when it reaches larger sizes?

So there are two issues.

1. Concern over the approach of using a single file for the entire
   database.  One of the benefits of ZODB is an open storage interface.
   This makes it possible to explore and implement the physical storage
   for the Zope database in a number os ways. So, if someone felt that 
   the database should be stored in multiple files, or on top of
   some other storage manager, it is certainly possible and, if using
   an existing storage manager (e.g. RDBMS, BDB), even straightforward.

2. Separating data into multiple databases for organizational reasons
   and combining the multiple databases into a single logical object
   space. This is doable too and something we plan to do, however, 
   it's harder than one might expect if one allows (non-trivial)
   cross-database object references.  If someone is interested in 
   working on this, possibly with simpler assumptions (like only
   supporting trivial cross-database references), contact me and
   I'll be happy to provide advice.

> I have this feeling it would totally rock.

Maybe, but I think that there are more direct ways to
address the issues above.  

Perhaps another approach that provides somewhat the same thing
is to provide an XML-RPC or SOAP interface to another Zope.
 
> Does anyone else think this would be cool, or have I just spent too much
> time behind the keyboard? :-) It is true ( I think) that it would likely
> need a ZODBMethod rather than an SQL method :-(

I think that there are already XML-RPC methods of some sort. Wouldn't
this, and multiple Zopes give you what you want?
 
> Of course, continuing the idea can lead to other cool concepts ....
> Roxen has the idea of mounted filesystems, such that you can have to
> physically seperate filesystems on the machine, mapping under each other
> ie. ...
> 
> /foo  (is /home/users/foo)
> /foo/ftp (is /home/ftp/users/foo)
> 
> ...and all requests are dealt with accordingly, no Regex'ing needed. It
> strikes me as useful that, given the above, one could carry it a step
> further, and have a 'directory' be a seperate ZODB. Now, _that_ would be
> cool. Alas, I have doubts as to whether this would be a less than
> massive undertaking. But I like the idea :-)

See issue 2 above.  I wouldn't call it massive, but I wouldn't
call it easy either.  

> As I said, the forst Idea has been percolating in my head for a few
> weeks now, had to get it out somewhere :-)

Whoooo. I bet that feels better. :)

> ... The second, well, it just
> came up ... just random musings ...

It is on the long-term list of things to do.


Jim

--
Jim Fulton           mailto:jim@digicool.com
Technical Director   (888) 344-4332              Python Powered!
Digital Creations    http://www.digicool.com     http://www.python.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.