[Zope-dev] Alternative Storages: (was RelationalStorage (was LocalFS))

[Jimmie Houchin]

This was on the Zope list and I've move this to Zope-dev as I feel it is
more appropriate.

I have been making notes to myself for awhile with ideas about things I
would like to explore with Zope Storages when I am able. I have not
posted any of them since talk is cheap. But I guess since I'm full of
cheap talk I'll let it go. :)

I'll also post this to the ZODB zwiki after I see what's left standing
after analysis. :)

As far as usage is concerned I generally like the ZODB best because it
is reasonably transparent to the building of a web app with Zope.
However there are areas in which it does not currently excel that which
if your site requires these skills then alternatives must be used. A
couple of areas are data size and heavy writes.

Some people use an RDBMS to solve these issues. While this will work it
does expect more from the developer. Some do not have the skills or the
tools. Even if one does it still requires leaving the transparency of
developing with the ZODB.

Multiple file storage for the ZODB has been proposed as a solution and
there are 2 proposals currently on the ZODB ZWiki. I will add another.

Class/Object based db files.

Each class gets it's own db file. This could be similar to the current
ZODB file except specific to a class. As objects are created they are
appended to the db file for their class. This could be somewhat
analogous to tables in an RDBMS.

Advantages would be spreading out the data space over multiple files
which would help with some oses. Also I think that each class has
different characteristics which would be able to be managed better if
separate.

Example: 
AutoParts
You have an AutoParts class. The objects will change very little once
created. However there are a lot of objects and news added periodically.
This file will need packed seldom. It will also be simple to backup and
not need backed up often as changes are periodic and regular.

RetailStore
In a retail store the product objects are very volatile. Vendors can
change. Prices do change. A productObject file would have different
usage characteristics than the AutoParts object.

Some classes are perfect for few writes and many reads. Others less so.

Earlier Andrew Kuchling was wanting to walk the object tree. This would
provide a relatively easy way to walk the object tree.

This could be implemented with some support classes which have to be
inherited from to create a class.db file. Any class not so doing would
go into the standard ZODB. This could help provide desired management
features for the characteristics of each. It would be nice if in the
management you could set the path to the file. This would allow for
multiple disks or partitions for data storage. This too would help with
backups and such.

Just a few ideas. They may not stand up to examination, but that's okay.
I just thought I would put them on the table.

Jimmie Houchin

 

Chris McDonough wrote:
> 
> All,
> 
> I've put some stuff about a proposed RelationalStorage in the ZODB wiki
> at http://www.zope.org/Members/jim/ZODB
> 
> Its goal is to allow you to use a SQL database as a Storage, which
> somewhat coincidentally would also get around the single-file 2GB
> limitation.
> 
> I would appreciate comments (in the wiki or here).  The table structure
> I'm not sure on, it's only a skeleton right now...
> 
> Jason Spisak wrote:
> >
> > Jonothan:
> >
> > > Shouldn't be too difficult. (I know, famous last words.) I'd be interested in
> > > banging out a prototype.
> >
> > I looked at FileStorage and the BasicStorage yesterday.  I am trying to get
> > a feel for it.
> >
> > > I'll have to see when I can get to it after responding
> > > to all these emails about LocalFS that piled up while I was on vacation.
> > >
> >
> > That's because it's an amazing product.
> >
> > All my best,
> >
> > Jason Spisak
> > CIO
> > HireTechs.com
> > 6151 West Century Boulevard
> > Suite 900
> > Los Angeles, CA 90045
> > P. 310.665.3444
> > F. 310.665.3544
> >
> > Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
> > address may not be added to any commercial mail list with out my
> > permission.  Violation of my privacy with advertising or SPAM will
> > result in a suit for a MINIMUM of $500 damages/incident, $1500 for
> > repeats.
> >
> > _______________________________________________
> > Zope maillist  -  Zope@zope.org
> > http://lists.zope.org/mailman/listinfo/zope
> > **   No cross posts or HTML encoding!  **
> > (Related lists -
> >  http://lists.zope.org/mailman/listinfo/zope-announce
> >  http://lists.zope.org/mailman/listinfo/zope-dev )
> 
> --
> Chris McDonough
> Digital Creations
> Publishers of Zope - http://www.zope.org
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )