[Zope3-dev] zodb __parent__ and weakref

[Shaun Cutts]

Opps,

The code I said would be attached...



> -----Original Message-----
> From: zope3-dev-bounces+shaun=cuttshome.net at zope.org
[mailto:zope3-dev-
> bounces+shaun=cuttshome.net at zope.org] On Behalf Of Shaun Cutts
> Sent: Friday, February 24, 2006 10:18 PM
> To: 'Chris Withers'; shaunc at speakeasy.net; 'zope3-dev'
> Subject: RE: [Zope3-dev] zodb __parent__ and weakref
> 
> Chris,
> 
> > You need to get into the habbit of CC'ing in the list you're
replying
> > too...
> 
> Sorry -- I was thinking that, if this was not a problem w/ ZODB and
> external objects, but I've missed something in implementation, maybe I
> should have been communicating on Zope3-users instead. Now I don't
know
> if I've communicated clearly enough to conclude this yet.
> 
> > shaunc at speakeasy.net wrote:
> > > The data is in Postgres, but I already have a python data
> > marshalling/demarshalling/business logic used by other processes, so
I
> > want to use this interface with ZOPE as well.
> >
> > Okay, and what is this interface?
> 
> Do you want to see the *whole* interface ? :) I would think it would
> suffice to say: it has interfaces to query the database and return
> python objects constructed from the data, and various "business logic"
> interfaces that trigger creation of new data in the database.
> 
> Zope is only one client. Others will be interacting with the database.
> 
> > > Should I be trying to inherit from Persistent rather than
> __getstate__,
> > __setstate__? ... perhaps there is a way as flagging a member
> volatile?...
> > Or that I want to control refresh myself?
> >
> > I don't see what implementing IContainer has to do with Persistent,
> > __getstate__ or __setstate__... or am I missing something?
> >
> I'm only implementing a prototype now, but in production, the database
> will have >=30 million objects in it after 2 years of use. For this
> reason, and because the db will be changing outside, it doesn't seem
> sensible to store the db objects in ZODB. This was also the opinion in
> zope3-users when I asked about this earlier.
> 
> Instead I want to have a container which queries the database for data
> itself, but avoids having these objects be put in the zodb, even if it
> itself is in the zodb.
> 
> You are saying that it isn't a good approach to implement __getstate__
> and __setstate__... but I don't see why this should be so? Can anyone
> explain why they are called so often? It seemed from other threads
that
> the danger would rather be that your data doesn't get stored at all if
> you don't explicitly set _p_changed (or whatever its called)....
> 
> I've attached a version of my attempted implementation (cleaned up
from
> the post in <[Zope3-Users] newbie design questions for UI to external
> data>.
> 
> 
> > > I thought that overriding __getstate__ and __setstate__ simply
> overrides
> > pythons default versions of these, which would be called just as
often
> as
> > my versions. I guess perhaps the default versions are especially
> efficient
> > because they just pass the instance dictionary back and forth(? --
but
> > then what does ZODB do with the instance dictionary? Why does it ask
> for
> > it if the object itself is ok?)
> >
> > Don't worry about any of these things, just don't try and implement
> > anything funky, you don't need to and you're likely to get hurt, as
> > you've found...
> >
> How should I implement this differently, or what bad assumptions am I
> making? To my mind, __getstate__ and __setstate__ are the normal way
to
> control persistence in pickle-using python, so "funky" is in the eye
of
> the beholder.
> 
> - Shaun
> 
> 
> 
> > Chris
> >
> > --
> > Simplistix - Content Management, Zope & Python Consulting
> >             - http://www.simplistix.co.uk
> >
> 
> 
> 
> _______________________________________________
> Zope3-dev mailing list
> Zope3-dev at zope.org
> Unsub: http://mail.zope.org/mailman/options/zope3-
> dev/shaun%40cuttshome.net
> 

-------------- next part --------------
class HollowContainer( IterableUserDict, Contained ):
    """
    Implements L{IHollowContainer}

    Strategy: for add and delete, notify the database directly; for
    update of contained object, however, we want to listen for
    ObjectModifiedEvent. This is better than getting a notification
    from the objects, as an edit form may set several fields, and then
    send the notification when done; so the event can be treated as a "commit"

    Keys: _iserial is key to objects in dictionary

    >>> from zope.interface.verify import verifyClass
    >>> verifyClass( IHollowContainer, HollowContainer )
    True

    """
    implements( IHollowContainer, INameChooser )
    
    # defaults for IHollowContainer
    database = None
    containedInterface = None
    
    # defaults for ILocation (base of IHollowContainer)
    __parent__ = None
    __name__ = None

    def __init__( self  ):
        super( HollowContainer, self ).__init__( )
        self.__setstate__()

    def __setitem__( self, key, obj ):
        "Implements L{cranedata.web.interfaces.IFundContainer.__setitem__}."
        if key in self.data:
            raise DuplicationError( "key %s already present" )
        
        self.add( obj )

    def add( self, obj ):
        obj = self.containedInterface( obj )
        object.__setattr__( obj, '__parent__', self )
        object.__setattr__( obj, '__name__', self.getKey() )
        
        setitem( self, self.data.__setitem__, self.getKey(), obj )
        self.database.add( obj )
        self.incrementKey()        

    def __delitem__( self, key ):
        obj = self.data[ key ]
        self.database.delete( obj )
        del self.data[ key ]
        uncontained( obj, self, obj.__name__ )

    # ok -- deal with update events:
    def update( self, obj ):
        assert obj.__parent__ == self
        self.database.update( obj )


    # support ZODB persistence:
    # __getstate__, __setstate__ need to save info stored
    # at object about its context, but should look "hollow"
    # from the point of view of the DB
    
    def __getstate__( self ):
        # we have to remember our tree-location
        dct = {
            '__name__' : self.__name__,
            '__parent__':self.__parent__
            }
        if hasattr( self, '__annotations__' ):
            dct[ '__annotations__' ] = self.__annotations__
            
        return dct

    def __setstate__( self, dct = None ):
        if dct:
            self.__parent__ = dct[ '__parent__' ]
            self.__name__ = dct[ '__name__' ]
            if '__annotations__' in dct:
                self.__annotations__ = dct[ '__annotations__' ]
            self.data = {}
            
        self.database.connect()
        self._load()
        
    def _load( self ):
        self.resetKey()
        hollowDispatcher.register( self )
        for row in self.database:
            self.data[ self.getKey() ] = row
            object.__setattr__( row, '__parent__', self )
            object.__setattr__( row, '__name__', self.getKey() )
            directlyProvides( row, IContained )
            self.incrementKey()

    def __del__( self ):
        hollowDispatcher.deregister( self )

    def getKey( self ):
        return unicode( self._iserial )

    def incrementKey( self ):
        self._iserial += 1L

    def resetKey( self ):
        self._iserial = 0L

    def checkName( self, name, object ):
        return type(name) == long

    def chooseName( self, name, object ):
        return self.getKey()

class HollowDispatcher:
    """a single instance of this class is used to catch
    event notifications for hollow containers.
    """

    def __init__( self ):
        self.registry = []
        
    def register( self, container ):
        if container not in self.registry:
            self.registry.append( container )

    def deregister( self, container ):
        self.registry.remove( container )

    def __call__( self, event ):
        try:
            parent = event.object.__parent__
        except AttributeError:
            return
        
        for container in self.registry:
            if parent is container:
                container.update( event.object )

hollowDispatcher = HollowDispatcher()