[Zope] persistence and dictionaries
Chris McDonough
chrism@digicool.com
Fri, 8 Dec 2000 14:57:01 -0500
Huh. If they do, it's by chance only. I'd be hard-pressed to explain it.
Do they inherit from *anything*?
----- Original Message -----
From: "Matt" <matt.bion@eudoramail.com>
To: "Chris McDonough" <chrism@digicool.com>
Cc: <zope@zope.org>
Sent: Friday, December 08, 2000 2:30 PM
Subject: Re: [Zope] persistence and dictionaries
> Chris, this was my original confusion .... the two places below where you
say
>
> "You can put instances which do not inherit from Persistence.Persistent in
> your database. They just won't "stick". They'll hang around until the
> server is restarted or for an undetermined amount of time during normal
> operations."
>
> "No. It'll work for "a while" or until the server is restarted. :-)"
>
> actually do persist after restarts ... that's what confused me, they
wouldn't
> go away and they should!!
>
> regards
> Matt
>
> Chris McDonough wrote:
>
> > > Thanks for the reply, that is really useful. There are a couple of
things
> > > though that still don't add up. Firstly, you say below, as do all the
> > ZODB
> > > documents that "Custom" classes can certainly persist, they just need
to
> > mix in
> > > the "Persistence.Persistent" class as a base class. Well, in my
example I
> > > attached in my first email, my product certainly has
> > Persistence.Persistent,
> > > but my second class that I add to this one does not, yet it still
> > persists.
> > > There was an email sometime ago on the mailing list that told someone
that
> > this
> > > was why their product instances disappearing from the ZODB.
> > > (the ref for the original email is :
> > http://www.egroups.com/message/zope/44263
> > > ... I can't find the reply again.)
> > >
> >
> > You can put instances which do not inherit from Persistence.Persistent
in
> > your database. They just won't "stick". They'll hang around until the
> > server is restarted or for an undetermined amount of time during normal
> > operations.
> >
> > > So my current understanding would be that any classes you want to add
in
> > do not
> > > need to derive from Persistence.Persistent, and if it is pickleable
then
> > all
> > > should be fine if you call on instances of that object within you
product.
> >
> > No. It'll work for "a while" or until the server is restarted. :-)
> >
> > > The next part that worried me came from the "python product tutorial"
> > > http://www.zope.org/Members/hathawsh/PythonProductTutorial
> > >
> > > This stated that the class dictionary self.votes = {} needed to be
changed
> > to
> > > self._votes = Globals.PersistentMapping() so that updates to it
persist.
> > > Hence my query about dictionaries.
> >
> > This was for convenience, I'd imagine.
> >
> > > I also noticed your comment about __setstate__ . What is it about
this
> > that is
> > > dangerous.
> >
> > Nothing implicitly dangerous, but it can get confusing if you have
multiple
> > revisions of your product and you use variables caused by __setstate__.
> > Also, once you add a __setstate__ which modifies the object in-place,
> > there's a likelihood that it can never go away (you're can never be sure
if
> > all instances have been updated).
> >
> > > Recently I built a product out of some python classes I wrapped
> > > around 4DOM, and since 4DOM documents do not seem to persist(well the
> > document
> > > does, but it loses all its children), then I persisted them to the
local
> > file
> > > system, since I needed to do that anyway for what I was doing.
Setstate
> > seemed
> > > to work nicely to bring them back, though watching its behaviour I
noticed
> > that
> > > it was called very often by zope.
> >
> > Sure, that works... although at that point you're creating your own
object
> > database. :-)
> >
> > >
> > > Chris McDonough wrote:
> > >
> > > > All pickleable Python primitive types (strings, dictionaries, lists,
> > Nones,
> > > > integers, floats, longs, etc.) can live in the ZODB. They can
persist
> > just
> > > > like instances that inherit from the Persistent class.
> > > >
> > > > I think you read a little too much in to the fact that you need to
> > "treat
> > > > mutable objects immutably" when working with them in the ZODB. This
> > > > statement doesn't mean that these kinds of objects can't be saved in
the
> > > > ZODB, it just means you need to treat them specially when putting
them
> > in
> > > > the database.
> > > >
> > > > For instance, if you were doing this inside of an external method:
> > > >
> > > > def amethod(self):
> > > > self.mydict = {}
> > > > self.mydict['a'] = 1
> > > >
> > > > (where self is the persistent object that is usually the external
> > method's
> > > > "container")
> > > >
> > > > It wouldn't work as you expected. Although you'd see an 'a' in
mydict
> > for a
> > > > little while in further accesses to it, 'mydict' would eventaully
show
> > up as
> > > > an empty dictionary on the first access of it after it was expired
from
> > the
> > > > RAM cache (after it was 'ghosted'), because the last thing that the
ZODB
> > > > "saw" (via the __setattr__ on 'self' and a subsequent transaction)
was
> > you
> > > > setting a empty dictionary.
> > > >
> > > > Persistent objects (like "self" in the above example) are only smart
> > enough
> > > > to notice changes to themselves that happen through their
__setattr__
> > (e.g.
> > > > self.mydict = {} calls self's __setattr__). Mutating the attribute
> > 'mydict'
> > > > above "in-place" (via self.mydict['a'] = 1) does not trigger self's
> > > > __setattr__, so the ZODB never notices that "mydict" got changed.
> > > >
> > > > There are two ways to handle this. The first is to treat mutable
> > attributes
> > > > "immutably" via assigning to a temporary variable and then making
sure
> > the
> > > > persistent container's __setattr__ gets called:
> > > >
> > > > def amethod(self):
> > > > dict = {}
> > > > dict['a'] = 1
> > > > self.mydict = dict # trigger persistence mechanism implicitly
> > > >
> > > > The second is to use the _p_changed attribute of the persistent
object
> > on
> > > > which the primitive is set. This explcitly tells the persistence
system
> > to
> > > > include the object on which it's set into the current transaction:
> > > >
> > > > def amethod(self):
> > > > self.mydict = {}
> > > > self.mydict['a'] = 1
> > > > self._p_changed = 1 # trigger persistence mechanism manually
> > > >
> > > > Variations on this theme extend to list methods too (e.g.
list.append,
> > > > list.pop, etc.)
> > > >
> > > > "Custom" classes can certainly persist, they just need to mix in the
> > > > "Persistence.Persistent" class as a base class.
> > > >
> > > > As long as you obey this (slightly annoying, but necessary) rule,
you
> > > > shouldn't need to use PersistentMapping (it's really just a
convenient
> > > > wrapper that does this "magic" for you) or make any other wrappers
for
> > other
> > > > mutable objects.
> > > >
> > > > I don't know why you're using __setstate__, but ummmm.. I won't go
> > there.
> > > > :-) I didn't look at your product, but I don't think I need to to
> > answer
> > > > your question...
> > > >
> > > > > Hi I am trying to get a handle on how I should handle peristence
in my
> > > > > python products. I have read all the ZODB documents and all the
> > Product
> > > > > tutorials, which all led me to believe that 1) mutable objects
such as
> > > > > lists and dictionaries are not persistent and that updates to
these
> > will
> > > > > not be implicitly saved into the ZODB, and 2) that custom classes
> > would
> > > > > certainly not persist. That all seemed fine, and I used persitent
> > > > > mapping and __setstate__ to fill things back in where necessary.
But
> > > > > then I decided to demonstrate that persitence does break as
suggested.
> > > > >
> > > > > I used boring product, added a dicitonary and a custom class that
> > > > > contains it's own dictionary.... let the user update name : value
> > pairs
> > > > > for both, and print the contents through index.dtml
> > > > >
> > > > > The problem is that everything persists fine !!!! through
restarts,
> > > > > everything?
> > > > >
> > > > > Why does it work??? shouldn't it not?
> > > > >
> > > > > I have attached the modified boring product .... boringplus .....
it's
> > > > > dead simple to follow if you have made products before.
> > > > >
> > > > > Any explanation would be really nice.
> > > > >
> > > > > regards
> > > > > Matt
> > > > >
> > > > >
> > >
> > >
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )
>
>