[Zope-dev] Alternative Storages: (was RelationalStorage (was LocalFS))

[Phillip J. Eby]

At 05:27 PM 5/3/00 -0500, Jimmie Houchin wrote:
>
>As far as usage is concerned I generally like the ZODB best because it
>is reasonably transparent to the building of a web app with Zope.
>However there are areas in which it does not currently excel that which
>if your site requires these skills then alternatives must be used. A
>couple of areas are data size and heavy writes.

Agreed.


>Some people use an RDBMS to solve these issues. While this will work it
>does expect more from the developer. Some do not have the skills or the
>tools. Even if one does it still requires leaving the transparency of
>developing with the ZODB.

Ty and I are working on fixing that.  As of today, the Specialist+Rack
subsystems of the ZPatterns library are now capable of managing objects
with mixed-source attributes and property sheets.  That is, objects stored
in a Rack can have attributes and property sheets whose data comes from
non-ZODB sources.  Currently, no non-ZODB sources are implemented, alas,
and the mechanism is only applicable to rack-mounted objects.  In future
releases, we'll be adding SQL and LDAP data sources, and the ability for
any Zope object to take advantage of these abilities (as long as it
subclasses RackMountable, and knows how to find the "designated Rack" which
will provide support for the desired features).


>Class/Object based db files.
>
>Each class gets it's own db file. This could be similar to the current
>ZODB file except specific to a class. As objects are created they are
>appended to the db file for their class. This could be somewhat

>analogous to tables in an RDBMS.
>
>Advantages would be spreading out the data space over multiple files
>which would help with some oses. Also I think that each class has
>different characteristics which would be able to be managed better if
>separate.
>

Basically, what you've described could be considered an instance of the
Specialist and Rack patterns of the RIPP model, but with a less flexible
implementation.  That is, you can do what you just described using the
Specialist/Rack patterns, but what you described won't do everything
Specialist/Rack can (such as mixed-source data for the same object or a
sensible place to put class extent methods (e.g. find, listAll, that sort
of thing)).

The only thing missing from Rack today that would be needed to implement a
"ZODB per class" approach, is the ability to use an alternative
"persistence provider" for storage.  I'd like to see some discussion on a
mechanism for providing such things.  My thought is that they would sort of
look like SQL Connection objects: something you can add, that can be
acquired, and which Racks can select from a dropdown.  (The dropdown even
already exists in the about-to-be-released ZPatterns 0.3.0, it just is
empty except for an option to use the primary ZODB.)

The thing about these "persistence providers", though, is that they need to
provide some kind of root for each thing that wants to use them.  I'm
assuming each provider may be shared by more than one Rack or rack-like
object, that security constraints need to exist on who can create root
branches in these DB's, and that the branches need to be able to be removed
when their owners go away.  If there were a design (or implementation,
better yet) for gizmos like these, hooking up Racks to use them would be a
snap,  since all Racks need to do is store a single BTree.

(By the way, Racks can be written to store the "base" object data in an
RDBMS, db file, LDAP, or anything else that you can implment createItem(),
retrieveItem(), and deleteItem() methods for.)