[Zope] persistence and dictionaries
Jonothan Farr
jfarr@real.com
Fri, 8 Dec 2000 17:15:37 -0800
Oops. You snipped the contents of MoreStuff.addStuff(). No way to tell what's
going on without that!
--jfarr
----- Original Message -----
From: "Matt" <matt.bion@eudoramail.com>
To: "Jonothan Farr" <jfarr@real.com>; <zope@zope.org>
Sent: Friday, December 08, 2000 2:39 PM
Subject: Re: [Zope] persistence and dictionaries
> Ok, here are some of the offending bits out of my boringplus product
> .... which I can send again if people want. I have
> another product that keeps a dictionary of cookies so I can see how many
> times someone uses the back button to
> access the same forms page again. This worked as expected, where I had
> to issue an _p_changed=1 to get them to
> persist. But I am doing a similar thing below, or so I thought, once
> with a dictionary in the product and once in a
> dictionary within a class that does not inherit Persistent. This is
> what is confusing. Through restarts etc, everything
> perisists
>
>
> class Boring(
> OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
> Persistent, # Make us persistent. Yaah!
> Acquisition.Implicit, # Uh, whatever.
> AccessControl.Role.RoleManager # Security manager.
> ):
>
>
> def __init__(self, id, title=''):
> self._things = {}
> self._more = myClass.MoreStuff()
>
>
> def manage_edit(self, title, REQUEST=None):
> if REQUEST is not None:
> if REQUEST.has_key('thing'):
> self._things[REQUEST['thing']] = REQUEST['thing_value']
> self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
>
>
> def addToMore(self,name,value):
> self._more.addStuff(name,value)
> return ""
>
>
> class MoreStuff:
>
> def __init__(self):
> self._stuff = {'animal':'monkey'}
>
> def addStuff(self,stuff_name,stuff_item):
>
>
> On Sat, 09 Dec 2000, Jonothan Farr wrote:
> > Maybe I'm mistaken, but it seems like you can put an instance of an
> object that
> > doesn't inherit from Persistent into the ZODB just fine, but its
> contents won't
> > persist, so you'll always end up with a copy of the object as it was
> first added
> > to the database.
> >
> > --jfarr
> >
> > ----- Original Message -----
> > From: "Chris McDonough" <chrism@digicool.com>
> > To: <matt.bion@eudoramail.com>
> > Cc: <zope@zope.org>
> > Sent: Friday, December 08, 2000 11:57 AM
> > Subject: Re: [Zope] persistence and dictionaries
> >
> >
> > > Huh. If they do, it's by chance only. I'd be hard-pressed to
> explain it.
> > >
> > > Do they inherit from *anything*?
> > >
> > > ----- Original Message -----
> > > From: "Matt" <matt.bion@eudoramail.com>
> > > To: "Chris McDonough" <chrism@digicool.com>
> > > Cc: <zope@zope.org>
> > > Sent: Friday, December 08, 2000 2:30 PM
> > > Subject: Re: [Zope] persistence and dictionaries
> > >
> > >
> > > > Chris, this was my original confusion .... the two places below
> where you
> > > say
> > > >
> > > > "You can put instances which do not inherit from
> Persistence.Persistent in
> > > > your database. They just won't "stick". They'll hang around
> until the
> > > > server is restarted or for an undetermined amount of time during
> normal
> > > > operations."
> > > >
> > > > "No. It'll work for "a while" or until the server is restarted.
> :-)"
> > > >
> > > > actually do persist after restarts ... that's what confused me,
> they
> > > wouldn't
> > > > go away and they should!!
> > > >
> > > > regards
> > > > Matt
> > > >
> > > > Chris McDonough wrote:
> > > >
> > > > > > Thanks for the reply, that is really useful. There are a
> couple of
> > > things
> > > > > > though that still don't add up. Firstly, you say below, as do
> all the
> > > > > ZODB
> > > > > > documents that "Custom" classes can certainly persist, they
> just need
> > > to
> > > > > mix in
> > > > > > the "Persistence.Persistent" class as a base class. Well, in
> my
> > > example I
> > > > > > attached in my first email, my product certainly has
> > > > > Persistence.Persistent,
> > > > > > but my second class that I add to this one does not, yet it
> still
> > > > > persists.
> > > > > > There was an email sometime ago on the mailing list that told
> someone
> > > that
> > > > > this
> > > > > > was why their product instances disappearing from the ZODB.
> > > > > > (the ref for the original email is :
> > > > > http://www.egroups.com/message/zope/44263
> > > > > > ... I can't find the reply again.)
> > > > > >
> > > > >
> > > > > You can put instances which do not inherit from
> Persistence.Persistent
> > > in
> > > > > your database. They just won't "stick". They'll hang around
> until the
> > > > > server is restarted or for an undetermined amount of time during
> normal
> > > > > operations.
> > > > >
> > > > > > So my current understanding would be that any classes you want
> to add
> > > in
> > > > > do not
> > > > > > need to derive from Persistence.Persistent, and if it is
> pickleable
> > > then
> > > > > all
> > > > > > should be fine if you call on instances of that object within
> you
> > > product.
> > > > >
> > > > > No. It'll work for "a while" or until the server is restarted.
> :-)
> > > > >
> > > > > > The next part that worried me came from the "python product
> tutorial"
> > > > > > http://www.zope.org/Members/hathawsh/PythonProductTutorial
> > > > > >
> > > > > > This stated that the class dictionary self.votes = {} needed
> to be
> > > changed
> > > > > to
> > > > > > self._votes = Globals.PersistentMapping() so that updates to
> it
> > > persist.
> > > > > > Hence my query about dictionaries.
> > > > >
> > > > > This was for convenience, I'd imagine.
> > > > >
> > > > > > I also noticed your comment about __setstate__ . What is it
> about
> > > this
> > > > > that is
> > > > > > dangerous.
> > > > >
> > > > > Nothing implicitly dangerous, but it can get confusing if you
> have
> > > multiple
> > > > > revisions of your product and you use variables caused by
> __setstate__.
> > > > > Also, once you add a __setstate__ which modifies the object
> in-place,
> > > > > there's a likelihood that it can never go away (you're can never
> be sure
> > > if
> > > > > all instances have been updated).
> > > > >
> > > > > > Recently I built a product out of some python classes I
> wrapped
> > > > > > around 4DOM, and since 4DOM documents do not seem to
> persist(well the
> > > > > document
> > > > > > does, but it loses all its children), then I persisted them to
> the
> > > local
> > > > > file
> > > > > > system, since I needed to do that anyway for what I was doing.
>
> > > Setstate
> > > > > seemed
> > > > > > to work nicely to bring them back, though watching its
> behaviour I
> > > noticed
> > > > > that
> > > > > > it was called very often by zope.
> > > > >
> > > > > Sure, that works... although at that point you're creating your
> own
> > > object
> > > > > database. :-)
> > > > >
> > > > > >
> > > > > > Chris McDonough wrote:
> > > > > >
> > > > > > > All pickleable Python primitive types (strings,
> dictionaries, lists,
> > > > > Nones,
> > > > > > > integers, floats, longs, etc.) can live in the ZODB. They
> can
> > > persist
> > > > > just
> > > > > > > like instances that inherit from the Persistent class.
> > > > > > >
> > > > > > > I think you read a little too much in to the fact that you
> need to
> > > > > "treat
> > > > > > > mutable objects immutably" when working with them in the
> ZODB. This
> > > > > > > statement doesn't mean that these kinds of objects can't be
> saved in
> > > the
> > > > > > > ZODB, it just means you need to treat them specially when
> putting
> > > them
> > > > > in
> > > > > > > the database.
> > > > > > >
> > > > > > > For instance, if you were doing this inside of an external
> method:
> > > > > > >
> > > > > > > def amethod(self):
> > > > > > > self.mydict = {}
> > > > > > > self.mydict['a'] = 1
> > > > > > >
> > > > > > > (where self is the persistent object that is usually the
> external
> > > > > method's
> > > > > > > "container")
> > > > > > >
> > > > > > > It wouldn't work as you expected. Although you'd see an 'a'
> in
> > > mydict
> > > > > for a
> > > > > > > little while in further accesses to it, 'mydict' would
> eventaully
> > > show
> > > > > up as
> > > > > > > an empty dictionary on the first access of it after it was
> expired
> > > from
> > > > > the
> > > > > > > RAM cache (after it was 'ghosted'), because the last thing
> that the
> > > ZODB
> > > > > > > "saw" (via the __setattr__ on 'self' and a subsequent
> transaction)
> > > was
> > > > > you
> > > > > > > setting a empty dictionary.
> > > > > > >
> > > > > > > Persistent objects (like "self" in the above example) are
> only smart
> > > > > enough
> > > > > > > to notice changes to themselves that happen through their
> > > __setattr__
> > > > > (e.g.
> > > > > > > self.mydict = {} calls self's __setattr__). Mutating the
> attribute
> > > > > 'mydict'
> > > > > > > above "in-place" (via self.mydict['a'] = 1) does not trigger
> self's
> > > > > > > __setattr__, so the ZODB never notices that "mydict" got
> changed.
> > > > > > >
> > > > > > > There are two ways to handle this. The first is to treat
> mutable
> > > > > attributes
> > > > > > > "immutably" via assigning to a temporary variable and then
> making
> > > sure
> > > > > the
> > > > > > > persistent container's __setattr__ gets called:
> > > > > > >
> > > > > > > def amethod(self):
> > > > > > > dict = {}
> > > > > > > dict['a'] = 1
> > > > > > > self.mydict = dict # trigger persistence mechanism
> implicitly
> > > > > > >
> > > > > > > The second is to use the _p_changed attribute of the
> persistent
> > > object
> > > > > on
> > > > > > > which the primitive is set. This explcitly tells the
> persistence
> > > system
> > > > > to
> > > > > > > include the object on which it's set into the current
> transaction:
> > > > > > >
> > > > > > > def amethod(self):
> > > > > > > self.mydict = {}
> > > > > > > self.mydict['a'] = 1
> > > > > > > self._p_changed = 1 # trigger persistence mechanism
> manually
> > > > > > >
> > > > > > > Variations on this theme extend to list methods too (e.g.
> > > list.append,
> > > > > > > list.pop, etc.)
> > > > > > >
> > > > > > > "Custom" classes can certainly persist, they just need to
> mix in the
> > > > > > > "Persistence.Persistent" class as a base class.
> > > > > > >
> > > > > > > As long as you obey this (slightly annoying, but necessary)
> rule,
> > > you
> > > > > > > shouldn't need to use PersistentMapping (it's really just a
> > > convenient
> > > > > > > wrapper that does this "magic" for you) or make any other
> wrappers
> > > for
> > > > > other
> > > > > > > mutable objects.
> > > > > > >
> > > > > > > I don't know why you're using __setstate__, but ummmm.. I
> won't go
> > > > > there.
> > > > > > > :-) I didn't look at your product, but I don't think I need
> to to
> > > > > answer
> > > > > > > your question...
> > > > > > >
> > > > > > > > Hi I am trying to get a handle on how I should handle
> peristence
> > > in my
> > > > > > > > python products. I have read all the ZODB documents and
> all the
> > > > > Product
> > > > > > > > tutorials, which all led me to believe that 1) mutable
> objects
> > > such as
> > > > > > > > lists and dictionaries are not persistent and that updates
> to
> > > these
> > > > > will
> > > > > > > > not be implicitly saved into the ZODB, and 2) that custom
> classes
> > > > > would
> > > > > > > > certainly not persist. That all seemed fine, and I used
> persitent
> > > > > > > > mapping and __setstate__ to fill things back in where
> necessary.
> > > But
> > > > > > > > then I decided to demonstrate that persitence does break
> as
> > > suggested.
> > > > > > > >
> > > > > > > > I used boring product, added a dicitonary and a custom
> class that
> > > > > > > > contains it's own dictionary.... let the user update name
> : value
> > > > > pairs
> > > > > > > > for both, and print the contents through index.dtml
> > > > > > > >
> > > > > > > > The problem is that everything persists fine !!!! through
>
> > > restarts,
> > > > > > > > everything?
> > > > > > > >
> > > > > > > > Why does it work??? shouldn't it not?
> > > > > > > >
> > > > > > > > I have attached the modified boring product ....
> boringplus .....
> > > it's
> > > > > > > > dead simple to follow if you have made products before.
> > > > > > > >
> > > > > > > > Any explanation would be really nice.
> > > > > > > >
> > > > > > > > regards
> > > > > > > > Matt
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Zope maillist - Zope@zope.org
> > > > http://lists.zope.org/mailman/listinfo/zope
> > > > ** No cross posts or HTML encoding! **
> > > > (Related lists -
> > > > http://lists.zope.org/mailman/listinfo/zope-announce
> > > > http://lists.zope.org/mailman/listinfo/zope-dev )
> > > >
> > > >
> > >
> > >
> > > _______________________________________________
> > > Zope maillist - Zope@zope.org
> > > http://lists.zope.org/mailman/listinfo/zope
> > > ** No cross posts or HTML encoding! **
> > > (Related lists -
> > > http://lists.zope.org/mailman/listinfo/zope-announce
> > > http://lists.zope.org/mailman/listinfo/zope-dev )
> > >
> >
> >
> > _______________________________________________
> > Zope maillist - Zope@zope.org
> > http://lists.zope.org/mailman/listinfo/zope
> > ** No cross posts or HTML encoding! **
> > (Related lists -
> > http://lists.zope.org/mailman/listinfo/zope-announce
> > http://lists.zope.org/mailman/listinfo/zope-dev )
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )
>