[Zope-dev] Re-inventing ZPatterns? (was Experiments with ORMapping)

[Shane Hathaway]

On Mon, 14 May 2001, Phillip J. Eby wrote:

> At 04:13 PM 5/14/01 -0400, Shane Hathaway wrote:
> >Regarding performance, this method is actually ideal IMHO.  Data is read
> >once, converted to an object, and kept for later connections, just like
> >ZODB.
>
> [shrug] Not any different than ZPatterns, except that Racks drop their
> caches after every transaction to ensure freshness (since few RDBMS have
> asynchronous cache invalidation messages available).

Hey, that would actually work here too.  Good idea!

> Of course, one can
> always tell the SQL methods to keep their results longer.  Of course,
> ZPatterns doesn't always "read data once" - it often reads data *zero*
> times, if it's not needed during that transaction.  Ty and I have written
> applications where certain data was in LDAP and other data in SQL, and
> either one or the other was needed for most transactions.  So ZPatterns
> only loads the attribute sets you need during that transaction.

This new scheme already does the same, but the approach is to create
"ghosts" instead of using hooks.  Ghosts have proven to be far more
reliable than __getattr__ hooks.  The proof of their reliability is in the
fact that programmers are very rarely aware of them.

> >Also, Jim has suggested database-specific features for querying, etc.
> >and abstracting these does require application-level logic.  The goal is
> >not to avoid all application logic, just to move as much as possible to
> >the storage layer.  (My statement about this approach not using any
> >application logic was thus inaccurate.)
>
> Which is why I think all you're going to end up with is an alternative
> implementation of what ZPatterns already does.  :)  I'm not opposed to that
> (not that it would mean anything if I were!), but I'd suggest taking a
> closer look at what's been done in that arena before investing a lot of
> time in it.  (May I suggest especially taking a look at SkinScript's
> provisions for storage and retrieval from *any* database, not just RDMBS'?)

The primary goal is RDBMS integration, but the code will be reusable for
other data access.

> Anyway, if you can improve on it and put it into Zope proper, great!  I'm
> off the hook for new ZPatterns releases.  :)  But I think all you're really
> proposing compared to ZPatterns is to:
>
> 1) Simplify Racks' attribute loading to compute all attributes at once
> (i.e. dropping ZPatterns' load-on-demand attribute groups)

More fundamentally, developers don't have to learn about "racks". :-)

> 2) Replace the Specialist-per-Interface approach of ZPatterns with a
> monolithic connection object which has (directly or indirectly) knowledge
> of storage rules for all application classes (i.e. reducing information
> hiding and composability of separate application components)

I didn't mean to give that impression.  No monolithic connection objects.
The experimental code in orconn.tar.gz shows what I have in mind for now.
Some slight changes to cPersistence would make it a lot cleaner.

> 3) Move the implementation of mapping rules, etc. out of the
> ZODB/through-the-web editing approach of ZPatterns and into Python code
> (Not sure on this one, you've been kind of vague about this part)

Not exactly.  It's easier to write Python code first, that's all.

> 4) Maybe refine the caching compared to ZPatterns so that data can hang
> around longer than per-transaction - but only if you can get invalidation
> messages from the source DB, which is a nontrivial exercise in
> itself.  (You might be able to create your own protocol, however, for
> broadcasting private invalidation messages among peers.)

We want to open up this possibility, yes.

> (Interestingly, the above are all directions which we originally thought to
> go with ZPatterns, or attempted in early drafts, but abandoned for reasons
> mostly implied above.)

You don't have the same goals we do.  You pursued the thing that was right
for you.

> 5) Use an API which emulates the existing persistence interface to the
> extent possible - which for most applications will basically mean that you
> will be able to set and get attributes the normal Python way - as long as
> they're relatively simple attributes.  (Same as ZPatterns does.)  For more
> complex things, things will be more complex.  (Again, same as with ZPatterns.)

Attributes will work just like they do in ZODB.  Objects are transparently
"de-ghostified" when attributes are accessed.  It seems to me that
ZPatterns makes this process complex by managing each attribute
individually through the use of __getattr__(), which this approach does
not need.  I could be misunderstanding your use of "complex".

Shane