[Zope-dev] Re-inventing ZPatterns? (was Experiments with ORMapping)

[Phillip J. Eby]

At 04:13 PM 5/14/01 -0400, Shane Hathaway wrote:
>"Phillip J. Eby" wrote:
> > All this is of course just my opinion, and quite possibly wrong.  But I
> > think that an application developer would get more mileage out of your
> > DBAPI product (coupled with some utility classes to replace Racks) or from
> > ZPatterns than they would from this approach to O-R mapping.
>
>Your opinion is highly valued, and I agree there are unknown quantities
>at this point.  But I think the unknowns are not significant enough to
>drop the idea.  The goal is transparent persistence with an arbitrary
>RDBMS schema.  Many developers prefer to code for ZODB but customers
>demand SQL.  This would bridge the gap.

The principal "unknown" is deciding which implementations will be used for 
which objects.  This is an implementation-level decision, but which 
requires domain-level knowledge.  You will find, I think, that this 
necessarily implies an "application framework" which deals with these 
decisions.  You may choose different ways of specifying and patterning 
these decision idioms than ZPatterns, but you will likely end up covering 
the same decision territory if you seek the same kinds of capabilities.



>Regarding performance, this method is actually ideal IMHO.  Data is read
>once, converted to an object, and kept for later connections, just like
>ZODB.

[shrug] Not any different than ZPatterns, except that Racks drop their 
caches after every transaction to ensure freshness (since few RDBMS have 
asynchronous cache invalidation messages available).  Of course, one can 
always tell the SQL methods to keep their results longer.  Of course, 
ZPatterns doesn't always "read data once" - it often reads data *zero* 
times, if it's not needed during that transaction.  Ty and I have written 
applications where certain data was in LDAP and other data in SQL, and 
either one or the other was needed for most transactions.  So ZPatterns 
only loads the attribute sets you need during that transaction.



>Also, Jim has suggested database-specific features for querying, etc.
>and abstracting these does require application-level logic.  The goal is
>not to avoid all application logic, just to move as much as possible to
>the storage layer.  (My statement about this approach not using any
>application logic was thus inaccurate.)

Which is why I think all you're going to end up with is an alternative 
implementation of what ZPatterns already does.  :)  I'm not opposed to that 
(not that it would mean anything if I were!), but I'd suggest taking a 
closer look at what's been done in that arena before investing a lot of 
time in it.  (May I suggest especially taking a look at SkinScript's 
provisions for storage and retrieval from *any* database, not just RDMBS'?)

Anyway, if you can improve on it and put it into Zope proper, great!  I'm 
off the hook for new ZPatterns releases.  :)  But I think all you're really 
proposing compared to ZPatterns is to:

1) Simplify Racks' attribute loading to compute all attributes at once 
(i.e. dropping ZPatterns' load-on-demand attribute groups)

2) Replace the Specialist-per-Interface approach of ZPatterns with a 
monolithic connection object which has (directly or indirectly) knowledge 
of storage rules for all application classes (i.e. reducing information 
hiding and composability of separate application components)

3) Move the implementation of mapping rules, etc. out of the 
ZODB/through-the-web editing approach of ZPatterns and into Python code 
(Not sure on this one, you've been kind of vague about this part)

4) Maybe refine the caching compared to ZPatterns so that data can hang 
around longer than per-transaction - but only if you can get invalidation 
messages from the source DB, which is a nontrivial exercise in 
itself.  (You might be able to create your own protocol, however, for 
broadcasting private invalidation messages among peers.)

(Interestingly, the above are all directions which we originally thought to 
go with ZPatterns, or attempted in early drafts, but abandoned for reasons 
mostly implied above.)

5) Use an API which emulates the existing persistence interface to the 
extent possible - which for most applications will basically mean that you 
will be able to set and get attributes the normal Python way - as long as 
they're relatively simple attributes.  (Same as ZPatterns does.)  For more 
complex things, things will be more complex.  (Again, same as with ZPatterns.)


Anyway, all that having been said, my Yahoo! horoscope said today that I 
should "Be proud, but don't be obnoxious," so I will stop here.  :)