[Zope-dev] Suggestions towards implementing an experimental protocol for cross-database persistent object references and multi-database Zopelications

[Phillip J. Eby]

At 10:43 AM 11/17/99 +0100, Martijn Pieters wrote:
>At 05:58 17/11/99 , Phillip J. Eby wrote:
>
>Also, IIRC, there is an issue with caching and connections, and the 
>question of when to close a connection to a referenced database. But as I 
>was experiencing some jet-lag at the time, and didn't bring a memo 
>recording when I spoke with Jim, I don't remember the full details.

Actually, this is (fairly) straightforward to handle since the connections
themselves are Persistent and subject to caching.  Implementing such a
connection as a Persistent object can be anywhere from very easy to
moderately complex, depending on the nature of the database backing it.
Manipulation of _p_changed may be necessary.  I point to ZLDAP as an
example of one that works, although I don't think it will currently work
correctly if the LDAPConnection is deactivated by the cache machinery
during a transaction.


>I believe that Jim Fulton has let his mind wander over this already. The 
>general implementation wasn't hard, he said, the problems arise in the 
>details. How, for example, do you remove unreferenced objects from any such 
>database? How do you find out if an otherwise unreferenced object in one 
>database, is not referenced by another one.

Like cross-database undo, cross-database garbage collection also requires a
global naming scheme.  However, the point of my proposal wasn't to create a
full multi-database Zope implementation, but rather, a tool for providing
database-independent referencing, making it possible for an object's place
in the Zope URL hierarchy to be independent of its physical database
location - a mechanism roughly analagous to a hard-link, but still
breakable like a symlink.  This hard-link mechanism improves over symlinks
by having a signifcantly lesser performance penalty for dereferencing,
better transparency of usage, and by allowing a reference to be independent
of the target object's acquisition hierarchy.  (Since it will be treated as
belonging to the hierarchy of the object that points to it, not the
hierarchy it originally came from.)

An example usage: suppose you create a UserFolder that lets you paste
arbitrary objects into it as users, and it stores them using
getReference().  You could, within the same folder, paste references to
users stored in relational databases, LDAP databases, and perhaps "person"
objects from some other part of the same ZODB.  The UserFolder wouldn't
care where they came from.  This might seem a contrived example, but in
dealing with some of the "legacy" environments I deal with, such a thing
could be quite handy in allowing only hand-selected individuals access to a
Zope installation, but without requiring password maintenance to be part of
the Zope application itself.