[Zope-dev] Experiments with ORMapping

[Shane Hathaway]

On Mon, 14 May 2001, ender wrote:

> i want to thank DC (jim, shane, and paul) for inviting me to come to the new
> DC offices. i had a great time and learned a bunch... and met
> the BFDL.

It was good to talk with you!

> i gave a quick overview of the smartobjects design/framework and  jim and
> shane presented some ideas for an or mapping for the zodb.
>
> to be honest it seemed both extremely elegant and slightly disturbing at the
> sametime. I've grown accustomed to treating the ZODB as a magic black box,
> that just works. integrating an or layer at this low level disturbs me as it
> meant that application developers would need to tinker at the lowest level of
> the zope framework, plus the dangers that to properly support application
> development the zodb would need to be bloated with accessory apis. at the
> same time it offers a really elegant way to make the or mapping available to
> python programmers on a pure python level (which is something i'm also
> interested in).

Jim's ideas are often disturbingly elegant and difficult to understand all
at the same time. :-)

This reminds me of a recent slashdot thread that talked about a
photographer in the days of black and white film who had devised a scheme
for preserving color, and the article presented the results.  It was
disturbing for readers to realize that life in past centuries actually was
in color!

Programmers really wouldn't have to tinker with this layer.  They would
just write a schema and a little glue code that this layer understands.

> for both approaches there are some similiar problems that must be tackled,
> namely definition of the or mapping, generation of stub python classes,
> integration with zope notions (permission, etc). so hopefully discussion here
> will flesh out, to the point of making a choice clear.
>
> i'm primarily focused on doing the or mapping definition in xml, as its
> language neutral and allows me to leverage other tools (like torque
> java.apache.org/turbine/torque.html) to assist in application
> development/porting.
>
> i've thought about it for a few days, and while i'm not convinced completely
> that a zodb approach is the best method, i'm going to start work on the some
> of the support classes nesc. for moving forward, namely some of  the xml
> model/generators outlined in my design (embryonic)
> (http://www.zope.org/Members/k_vertigo/smartobjects)
>
> most of my concerns echo the issues that philip has brought up re the
> need of application developers. i do feel some if not most of these issues
> can be solved, i'm just wondering if for zope this is the best solution. for
> pure python development it seems alright. for zope, i wonder if a higher
> level application framework is more suitable.

It all depends on what layer you're developing for, I guess.  In the web
interface you can't use rich object hierarchies as fluidly as you can in
Python.  On the web level you don't care as much about transparent
persistence, but in Python you do.

> to clarify some existing points in this thread to date.
>
> - (IMO) zpatterns is about two things, delegation and data abstraction at
> the application level. the zpatterns framework doesn't assist in automating
> this and its up to the integrator/developer to manually wire together all the
> app/business objects. the smartobjects framework is about automating
> development and presenting easier development interfaces for both
> developers and designers. this isn't to say that such automation and
> interfaces couldn't be built on top of zpatterns, just to say that there are
> some different design goals.

Let's say you want CMF objects to be stored in an RDBMS.  You don't want
to use ZODB stubs and you want a tree of persistent objects.  ZPatterns
can't help you.  Can SmartObjects?

> - one of the primary goals of smartobjects is easy integration with zope in a
> manner that is natural for zope developers. this does not mandate a new db
> access api, its about facilitating automation of this api so that the objects
> can be treated as normaly zope objects. both approaches (smartobjects and or
> zodb) must still tackle similiar fundamental issues to integrate with zope.
> on the zodb level this seems complicated to me (which betrays my incomplete
> understanding of the zodb internals), plus it requires some additions to the
> zodb interface to allow effective application usage (namely a generic query
> mechanism, and probably interogation of a class information).

Here's another way to look at this: if SmartObjects doesn't use code from
the ZODB, it could be reinventing parts of it.  It seems like a good idea
to get to know ZODB well before proceeding too far.

> - one of my main objectives in developing an or mapping is to use the
> acs4 (www.arsdigita.com) applications natively (or as close as possible)
> within zope.

Sounds great!

> - my smartobjects design (www.zope.org/Members/k_vertigo/smartobjects) is
> mine, in that smartobjects is primarily a iuveno project this does not mean
> it is smartobjects. stephen richter (of iuveno) liked parts of the design and
> will likely use it parts of it in a iuveno implementation but what comes out
> of iuveno is not limited or beholden in anyway by my proposal. the key focus
> of the proposal is a metamodel (schema) based approach.

Take a look at my proposal and see how closely our goals match.

http://dev.zope.org/Wikis/DevSite/Proposals/ORMappingDB

> - i think any integration of or layer should be complementary to the zope
> framework. shane's existing examples and discussion of an or layer seem to be
> more geared towards replacing the zodb, than complementing. is the idea to
> use some sort of zodb product as a mounted storage?, or is this a replacement
> strategy.

Either or both. :-)  One could:

- store most objects in ZODB while putting certain pieces of the hierarchy
in the RDBMS,

- mount an RDBMS, or

- store objects for which a schema has not been written in a special
table, either pickled or exploded.

Shane