[ZODB-Dev] Proposal: cross database reference seat belt

[Jim Fulton]

Thanks for writing this up!

Do we really need per-object control?

It would be simpler to simply allow or disallow (new) cross-data  
references at the database level.

I think the most common use case for cross-database connection  
limitation is support for "mounts".  With mounts, there are a small  
number of direct references between databases.  This could be  
addressed by instantiating a database in a mode that allows cross- 
database references and using that instantiation to establish the  
mounts. All other instantiations would disallow new cross-database  
references.

Do you have a specific use case that requires _p_check_xref?

Jim

On Apr 28, 2009, at 5:31 AM, Shane Hathaway wrote:

> Problem
> -------
>
> In multi-database configurations, ZODB applications can  
> unintentionally create cross-database references.  The causes  
> include moving (rather than copying) an object between containers,  
> storing an object in a session database and later moving it to the  
> main database, and using a persistent object for a catalog index  
> when the catalog is located in another database.
>
> Unintentional cross-database references can cause significant  
> problems.  For example, references from non-volatile objects to  
> volatile session objects will break when the session expires,  
> leading to application errors.
>
> In a project I am working on, my team decided that configuring our  
> application to use a multi-database was too risky unless we had some  
> way to prevent unintentional cross-database references.
>
> Proposed Solution
> -----------------
>
> I propose an optional "seat belt" for cross-database references in  
> ZODB.  The seat belt, when enabled, will prevent most objects from  
> holding any cross-database references.  Application policy will  
> specify which cross-database references to allow.  When any cross- 
> database reference violates the policy, ZODB will raise an exception  
> to help application developers track down the policy violation.
>
> Proposed Mechanism
> ------------------
>
> The ZODB.DB.DB constructor will accept a new parameter,  
> "check_xrefs", that defaults to False.  When check_xrefs is True,  
> the cross-database reference seat belt is enabled.  "xref" is short  
> for "cross-database reference".
>
> Applications will express cross-database reference policy through a  
> new method of persistent objects named "_p_check_xref".  The  
> _p_check_xref method accepts a single parameter, the object to be  
> referenced in another database.  If the reference should be allowed,  
> the _p_check_xref method returns True.  If the _p_check_xref method  
> returns False, the object serialization machinery will raise an  
> InvalidObjectReference exception, leading to transaction abort.
>
> Furthermore, when the seat belt is enabled, ZODB will raise an  
> InvalidObjectReference exception when an object with no  
> _p_check_xref method attempts to hold a cross-database reference.
>
> Each database in a multi-database has its own check_xrefs setting.   
> The setting applies only to the objects contained in that database.   
> This allows developers to specify, for example, that arbitrary  
> references from the main database to the volatile session database  
> are disallowed, while arbitrary references from the volatile session  
> database to the main database are allowed.
>
> The proposed mechanism has been implemented in a branch of ZODB  
> named "shane-cross-database-seatbelt", checked in at svn.zope.org.   
> It does not change very many lines of code.
>
> Some possible risks of the proposed mechanism:
>
> * An implementation of the _p_check_xref method could wake up ghosts  
> as a side effect.  Implementers probably need to avoid that.
>
> * People might want to express the application policy without  
> modifying persistent classes.  I chose not to propose that kind of  
> solution, opting instead for a solution that follows established  
> ZODB patterns.
>
> Conclusion
> ----------
>
> The proposed new feature is designed to help developers create more  
> robust ZODB applications by enforcing cross-database reference  
> policy. I hope it can be included in ZODB 3.9.
>
> Shane
>

--
Jim Fulton
Zope Corporation