[ZODB-Dev] Proposal (version 2): cross database reference seat belt

[Jim Fulton]

Thanks again!

(Note to everyone else, Shane and I discussed this on IRC, along with  
another alternative that I'll mention below.)

I like version 2 better than version 1.  I'd be inclined to simplify  
and it and skip the configuration flag and simply publish an event any  
time we see a cross-database reference when saving an object.

Here's proposed solution 3. :)

- We add a flag to disable new cross-database references unless they  
are explicitly registered.
- We add a connection method to register a reference:

      def registerCrossDatabaseReference(from_, to):
            "Register a new cross-database reference from from_ to to."

- We arrange that connections can recognize old cross-database  
references.

If someone accidentally creates a new reference and the flag is set,  
then transaction will be aborted.

An interim step, if we're in a hurry to get 3.9 out, is to simply add  
the flag.  This would disallow cross-database references in new  
applications.  These applications could still support multiple  
databases by providing application-level traversal across databases.

Thoughts?

If there are no objections, I'll pursue version 3.  How soon do we  
want 3.9 beta?

Jim

On Apr 28, 2009, at 1:31 PM, Shane Hathaway wrote:

> Problem
> -------
>
> In multi-database configurations, ZODB applications can  
> unintentionally
> create cross-database references.  The causes include moving (rather
> than copying) an object between containers, storing an object in a
> session database and later moving it to the main database, and using a
> persistent object for a catalog index when the catalog is located in
> another database.
>
> Unintentional cross-database references can cause significant  
> problems.
> For example, references from non-volatile objects to volatile session
> objects will break when the session expires, leading to application  
> errors.
>
> In a project I am working on, my team decided that configuring our
> application to use a multi-database was too risky unless we had some  
> way
> to prevent unintentional cross-database references.
>
> Proposed Solution
> -----------------
>
> I propose an optional "seat belt" for cross-database references in  
> ZODB.
> The seat belt, when enabled, gives applications an opportunity to  
> veto the creation of cross-database references.  Application policy  
> will specify which cross-database references to allow.
>
> Proposed Mechanism
> ------------------
>
> The ZODB.DB.DB constructor will accept a new parameter, "check_xrefs",
> that defaults to False.  When check_xrefs is True, the cross-database
> reference seat belt is enabled.  "xref" is short for "cross-database
> reference".
>
> When the cross-database reference seat belt is enabled for a  
> database, ZODB publishes events (using the zope.event package) every  
> time an object that contains cross-database references changes.   
> Event subscribers have the opportunity to veto the cross reference  
> by generating an exception.
>
> The published event will conform to the following interface.
>
>    class ICrossDatabaseReferenceEvent(Interface):
>        source = Attribute("The object holding the reference")
>        target = Attribute("The target of the reference")
>
> Each database in a multi-database has its own check_xrefs setting.   
> The
> setting applies only to the objects contained in that database.  This
> allows developers to specify, for example, that arbitrary references
> from the main database to the volatile session database are  
> disallowed,
> while arbitrary references from the volatile session database to the
> main database are allowed.
>
> A possible risk of the proposed mechanism is that an event  
> subscriber could wake up ghosts as a side effect.  Implementers  
> probably need to avoid that.
>
> Conclusion
> ----------
>
> The proposed new feature is designed to help developers create more
> robust ZODB applications by enforcing cross-database reference policy.
> I hope it can be included in ZODB 3.9.
>
> Shane
>

--
Jim Fulton
Zope Corporation