[Zope] persistence and dictionaries

[Matt]

Chris, this was my original confusion .... the two places below where you say

"You can put instances which do not inherit from Persistence.Persistent in
your database.  They just won't "stick".  They'll hang around until the
server is restarted or for an undetermined amount of time during normal
operations."

"No.  It'll work for "a while" or until the server is restarted.  :-)"

actually do persist after restarts ... that's what confused me, they wouldn't
go away and they should!!

regards
Matt

Chris McDonough wrote:

> > Thanks for the reply, that is really useful.  There are a couple of things
> > though that still don't add up.  Firstly, you say below, as do all the
> ZODB
> > documents that "Custom" classes can certainly persist, they just need to
> mix in
> > the "Persistence.Persistent" class as a base class.  Well, in my example I
> > attached in my first email, my product certainly has
> Persistence.Persistent,
> > but my second class that I add to this one does not, yet it still
> persists.
> > There was an email sometime ago on the mailing list that told someone that
> this
> > was why their product instances disappearing from the ZODB.
> > (the ref for the original email is :
> http://www.egroups.com/message/zope/44263
> > ... I can't find the reply again.)
> >
>
> You can put instances which do not inherit from Persistence.Persistent in
> your database.  They just won't "stick".  They'll hang around until the
> server is restarted or for an undetermined amount of time during normal
> operations.
>
> > So my current understanding would be that any classes you want to add in
> do not
> > need to derive from Persistence.Persistent, and if it is pickleable then
> all
> > should be fine if you call on instances of that object within you product.
>
> No.  It'll work for "a while" or until the server is restarted.  :-)
>
> > The next part that worried me came from the "python product tutorial"
> > http://www.zope.org/Members/hathawsh/PythonProductTutorial
> >
> > This stated that the class dictionary self.votes = {} needed to be changed
> to
> > self._votes = Globals.PersistentMapping()  so that updates to it persist.
> > Hence my query about dictionaries.
>
> This was for convenience, I'd imagine.
>
> >  I also noticed your comment about __setstate__ .  What is it about this
> that is
> > dangerous.
>
> Nothing implicitly dangerous, but it can get confusing if you have multiple
> revisions of your product and you use variables caused by __setstate__.
> Also, once you add a __setstate__ which modifies the object in-place,
> there's a likelihood that it can never go away (you're can never be sure if
> all instances have been updated).
>
> > Recently I built a product out of some python classes I wrapped
> > around 4DOM, and since 4DOM documents do not seem to persist(well the
> document
> > does, but it loses all its children), then I persisted them to the local
> file
> > system, since I needed to do that anyway for what I was doing.  Setstate
> seemed
> > to work nicely to bring them back, though watching its behaviour I noticed
> that
> > it was called very often by zope.
>
> Sure, that works... although at that point you're creating your own object
> database.  :-)
>
> >
> > Chris McDonough wrote:
> >
> > > All pickleable Python primitive types (strings, dictionaries, lists,
> Nones,
> > > integers, floats, longs, etc.) can live in the ZODB.  They can persist
> just
> > > like instances that inherit from the Persistent class.
> > >
> > > I think you read a little too much in to the fact that you need to
> "treat
> > > mutable objects immutably" when working with them in the ZODB.  This
> > > statement doesn't mean that these kinds of objects can't be saved in the
> > > ZODB, it just means you need to treat them specially when putting them
> in
> > > the database.
> > >
> > > For instance, if you were doing this inside of an external method:
> > >
> > > def amethod(self):
> > >    self.mydict = {}
> > >    self.mydict['a'] = 1
> > >
> > > (where self is the persistent object that is usually the external
> method's
> > > "container")
> > >
> > > It wouldn't work as you expected.  Although you'd see an 'a' in mydict
> for a
> > > little while in further accesses to it, 'mydict' would eventaully show
> up as
> > > an empty dictionary on the first access of it after it was expired from
> the
> > > RAM cache (after it was 'ghosted'), because the last thing that the ZODB
> > > "saw" (via the __setattr__ on 'self' and a subsequent transaction) was
> you
> > > setting a empty dictionary.
> > >
> > > Persistent objects (like "self" in the above example) are only smart
> enough
> > > to notice changes to themselves that happen through their __setattr__
> (e.g.
> > > self.mydict = {} calls self's __setattr__).  Mutating the attribute
> 'mydict'
> > > above "in-place" (via self.mydict['a'] = 1) does not trigger self's
> > > __setattr__, so the ZODB never notices that "mydict" got changed.
> > >
> > > There are two ways to handle this.  The first is to treat mutable
> attributes
> > > "immutably" via assigning to a temporary variable and then making sure
> the
> > > persistent container's __setattr__ gets called:
> > >
> > > def amethod(self):
> > >    dict = {}
> > >    dict['a'] = 1
> > >    self.mydict = dict # trigger persistence mechanism implicitly
> > >
> > > The second is to use the _p_changed attribute of the persistent object
> on
> > > which the primitive is set.  This explcitly tells the persistence system
> to
> > > include the object on which it's set into the current transaction:
> > >
> > > def amethod(self):
> > >    self.mydict = {}
> > >    self.mydict['a'] = 1
> > >    self._p_changed = 1 # trigger persistence mechanism manually
> > >
> > > Variations on this theme extend to list methods too (e.g. list.append,
> > > list.pop, etc.)
> > >
> > > "Custom" classes can certainly persist, they just need to mix in the
> > > "Persistence.Persistent" class as a base class.
> > >
> > > As long as you obey this (slightly annoying, but necessary) rule, you
> > > shouldn't need to use PersistentMapping (it's really just a convenient
> > > wrapper that does this "magic" for you) or make any other wrappers for
> other
> > > mutable objects.
> > >
> > > I don't know why you're using __setstate__, but ummmm.. I won't go
> there.
> > > :-)  I didn't look at your product, but I don't think I need to to
> answer
> > > your question...
> > >
> > > > Hi I am trying to get a handle on how I should handle peristence in my
> > > > python products.  I have read all the ZODB documents and all the
> Product
> > > > tutorials, which all led me to believe that 1) mutable objects such as
> > > > lists and dictionaries are not persistent and that updates to these
> will
> > > > not be implicitly saved into the ZODB, and 2) that custom classes
> would
> > > > certainly not persist.  That all seemed fine, and I used persitent
> > > > mapping and __setstate__ to fill things back in where necessary.  But
> > > > then I decided to demonstrate that persitence does break as suggested.
> > > >
> > > > I used boring product, added a dicitonary and a custom class that
> > > > contains it's own dictionary.... let the user update name : value
> pairs
> > > > for both, and print the contents through index.dtml
> > > >
> > > > The problem is that everything persists fine !!!!  through restarts,
> > > > everything?
> > > >
> > > > Why does it work???  shouldn't it not?
> > > >
> > > > I have attached the modified boring product .... boringplus ..... it's
> > > > dead simple to follow if you have made products before.
> > > >
> > > > Any explanation would be really nice.
> > > >
> > > > regards
> > > > Matt
> > > >
> > > >
> >
> >