[Zope-dev] Experiments with ORMapping

[Shane Hathaway]

Joachim Werner wrote:
> 
> Hi!
> 
> > > Do you know that there already is a project for OR-Mapping in Zope
> (actually
> > > there are two ...)?
> >
> > Yes, and I think the projects need to look into replacing parts of ZODB
> > rather than adding complexity.  ZODB has pieces that can be split apart
> > and replaced as needed, such as caching, persistence, transactions, the
> > "pickle jar", the multi-threaded connection factory, and the storage
> > layer.  I'm hoping we can achieve OR mapping by only replacing the
> > "pickle jar", i.e. Connection.py.
> 
> The current design plans of SmartObjects are mainly based on the assumption
> that we will not be able to change Zope itself. This is not a dogma for us,
> however. I guess doing OR-mapping in the Zope core would be fine with us ;-)

FYI by replacing I don't mean changing anything in the Zope core.  I
mean using a different class in place of Connection, which you can do
just by creating a custom_zodb.py.

> But I am quite sure it will not solve ALL issues SmartObjects is going to
> tackle. One of the things we want to accomplish is "better ZClasses".
> Another thing is queries: You enumerated a couple of reasons why people
> would use SQL instead of ZODB. One is integration of legacy data of course.

It sounds like you are tackling a lot of things at once.  You might
consider dividing into separate projects.

> But the more important one for us is that we still believe that even with
> ZCatalog Zope can not really do efficient "croos-tree" queries in all cases.
> A query like "Give me all users who have bought this and that product and
> are aged 20 or above" can not be handled by the catalog, I think.

Yes it can. :-)  That's not to say that it has as much flexibility as a
SQL query, but it can do most of the things people usually do with one
table at a time.

Here's the goal I envision for OR mapping: to be able to move between
OODBMS and RDBMS seamlessly.  It makes sense to develop on top of ZODB
then move to an RDBMS for testing and deployment.

> Same with
> the appointment things in our groupware project: While it is trivial for SQL
> to return me all appointments all members of my project have on a specific
> day (security is handled by the SmartObjects folders' methods in that case),
> ZODB/ZCatalog would only be able to resolve this question if either all
> appointments are stored in a central folder (what about security then?) or
> everything is cataloged (again, the standard ZCatalog does not handle
> security here, right?)

The CMF has a solution for this, actually.  The results of a query
include only the things you're allowed to access.

> What I need is an object-oriented storage (and I agree that ZODB is one of
> the best of them) that can efficiently be queried "cross-tree". One
> different approach to this topic could be a better ZCatalog (maybe one that
> stores the catalog in relational tables).

Right.  I agree an *alternate* ZCatalog would be useful.

> > > "Ours" is SmartObjects
> > > (http://demo.iuveno-net.de/iuveno/Products/SmartObjects). Also see the
> > > mailing list archive at
> > > http://imail.iuveno-net.de/pipermail/smartobjects/.
> >
> > Kapil gave Jim and I a good introduction to SmartObjects yesterday.  So
> > far, it seems a lot like ZPatterns in that it mandates a new database
> > access API rather than trying to be transparent like ZODB.
> 
> I think whether SmartObjects (in the current design) can be called
> "transparent" depends on the level. On the user level, they will be. You can
> Add them to folders, cut&paste them, upload them via FTP etc. The Zope
> security API is also used. So they are much more transparent than direct
> connections to LDAP, SQL, or to the file system (o.k. for the filesystem
> extFile does this already). Some of the API will definitely be needed: A
> SmartObject will have to be connected to databases like SQL-DAs or
> LDAPAdapter have to. So there has to be a ZMI-Plugin, and there has to be an
> API for that.
> 
> SmartObjects is more of a programming framework than just adding OR-Mapping
> to Zope. So if we can solve the storage and query parts more efficiently by
> just having Zope itself extended a bit, this would be very cool ...

You see, I think it is not necessary to create a programming framework
if the goal is OR mapping.  The framework is already defined, and it's
Python / ZODB.  But SmartObjects seems to have many loosely related
goals, making it difficult to assist.

Shane