[Zope] persistence and dictionaries
Matt
matt.bion@eudoramail.com
Sat, 09 Dec 2000 11:39:13 +1300
Ok, here are some of the offending bits out of my boringplus product
.... which I can send again if people want. I have
another product that keeps a dictionary of cookies so I can see how many
times someone uses the back button to
access the same forms page again. This worked as expected, where I had
to issue an _p_changed=1 to get them to
persist. But I am doing a similar thing below, or so I thought, once
with a dictionary in the product and once in a
dictionary within a class that does not inherit Persistent. This is
what is confusing. Through restarts etc, everything
perisists
class Boring(
OFS.SimpleItem.Item, # A simple Principia object. Not Folderish.
Persistent, # Make us persistent. Yaah!
Acquisition.Implicit, # Uh, whatever.
AccessControl.Role.RoleManager # Security manager.
):
def __init__(self, id, title=''):
self._things = {}
self._more = myClass.MoreStuff()
def manage_edit(self, title, REQUEST=None):
if REQUEST is not None:
if REQUEST.has_key('thing'):
self._things[REQUEST['thing']] = REQUEST['thing_value']
self.addToMore(REQUEST['thing'],REQUEST['thing_value'])
def addToMore(self,name,value):
self._more.addStuff(name,value)
return ""
class MoreStuff:
def __init__(self):
self._stuff = {'animal':'monkey'}
def addStuff(self,stuff_name,stuff_item):
On Sat, 09 Dec 2000, Jonothan Farr wrote:
> Maybe I'm mistaken, but it seems like you can put an instance of an
object that
> doesn't inherit from Persistent into the ZODB just fine, but its
contents won't
> persist, so you'll always end up with a copy of the object as it was
first added
> to the database.
>
> --jfarr
>
> ----- Original Message -----
> From: "Chris McDonough" <chrism@digicool.com>
> To: <matt.bion@eudoramail.com>
> Cc: <zope@zope.org>
> Sent: Friday, December 08, 2000 11:57 AM
> Subject: Re: [Zope] persistence and dictionaries
>
>
> > Huh. If they do, it's by chance only. I'd be hard-pressed to
explain it.
> >
> > Do they inherit from *anything*?
> >
> > ----- Original Message -----
> > From: "Matt" <matt.bion@eudoramail.com>
> > To: "Chris McDonough" <chrism@digicool.com>
> > Cc: <zope@zope.org>
> > Sent: Friday, December 08, 2000 2:30 PM
> > Subject: Re: [Zope] persistence and dictionaries
> >
> >
> > > Chris, this was my original confusion .... the two places below
where you
> > say
> > >
> > > "You can put instances which do not inherit from
Persistence.Persistent in
> > > your database. They just won't "stick". They'll hang around
until the
> > > server is restarted or for an undetermined amount of time during
normal
> > > operations."
> > >
> > > "No. It'll work for "a while" or until the server is restarted.
:-)"
> > >
> > > actually do persist after restarts ... that's what confused me,
they
> > wouldn't
> > > go away and they should!!
> > >
> > > regards
> > > Matt
> > >
> > > Chris McDonough wrote:
> > >
> > > > > Thanks for the reply, that is really useful. There are a
couple of
> > things
> > > > > though that still don't add up. Firstly, you say below, as do
all the
> > > > ZODB
> > > > > documents that "Custom" classes can certainly persist, they
just need
> > to
> > > > mix in
> > > > > the "Persistence.Persistent" class as a base class. Well, in
my
> > example I
> > > > > attached in my first email, my product certainly has
> > > > Persistence.Persistent,
> > > > > but my second class that I add to this one does not, yet it
still
> > > > persists.
> > > > > There was an email sometime ago on the mailing list that told
someone
> > that
> > > > this
> > > > > was why their product instances disappearing from the ZODB.
> > > > > (the ref for the original email is :
> > > > http://www.egroups.com/message/zope/44263
> > > > > ... I can't find the reply again.)
> > > > >
> > > >
> > > > You can put instances which do not inherit from
Persistence.Persistent
> > in
> > > > your database. They just won't "stick". They'll hang around
until the
> > > > server is restarted or for an undetermined amount of time during
normal
> > > > operations.
> > > >
> > > > > So my current understanding would be that any classes you want
to add
> > in
> > > > do not
> > > > > need to derive from Persistence.Persistent, and if it is
pickleable
> > then
> > > > all
> > > > > should be fine if you call on instances of that object within
you
> > product.
> > > >
> > > > No. It'll work for "a while" or until the server is restarted.
:-)
> > > >
> > > > > The next part that worried me came from the "python product
tutorial"
> > > > > http://www.zope.org/Members/hathawsh/PythonProductTutorial
> > > > >
> > > > > This stated that the class dictionary self.votes = {} needed
to be
> > changed
> > > > to
> > > > > self._votes = Globals.PersistentMapping() so that updates to
it
> > persist.
> > > > > Hence my query about dictionaries.
> > > >
> > > > This was for convenience, I'd imagine.
> > > >
> > > > > I also noticed your comment about __setstate__ . What is it
about
> > this
> > > > that is
> > > > > dangerous.
> > > >
> > > > Nothing implicitly dangerous, but it can get confusing if you
have
> > multiple
> > > > revisions of your product and you use variables caused by
__setstate__.
> > > > Also, once you add a __setstate__ which modifies the object
in-place,
> > > > there's a likelihood that it can never go away (you're can never
be sure
> > if
> > > > all instances have been updated).
> > > >
> > > > > Recently I built a product out of some python classes I
wrapped
> > > > > around 4DOM, and since 4DOM documents do not seem to
persist(well the
> > > > document
> > > > > does, but it loses all its children), then I persisted them to
the
> > local
> > > > file
> > > > > system, since I needed to do that anyway for what I was doing.
> > Setstate
> > > > seemed
> > > > > to work nicely to bring them back, though watching its
behaviour I
> > noticed
> > > > that
> > > > > it was called very often by zope.
> > > >
> > > > Sure, that works... although at that point you're creating your
own
> > object
> > > > database. :-)
> > > >
> > > > >
> > > > > Chris McDonough wrote:
> > > > >
> > > > > > All pickleable Python primitive types (strings,
dictionaries, lists,
> > > > Nones,
> > > > > > integers, floats, longs, etc.) can live in the ZODB. They
can
> > persist
> > > > just
> > > > > > like instances that inherit from the Persistent class.
> > > > > >
> > > > > > I think you read a little too much in to the fact that you
need to
> > > > "treat
> > > > > > mutable objects immutably" when working with them in the
ZODB. This
> > > > > > statement doesn't mean that these kinds of objects can't be
saved in
> > the
> > > > > > ZODB, it just means you need to treat them specially when
putting
> > them
> > > > in
> > > > > > the database.
> > > > > >
> > > > > > For instance, if you were doing this inside of an external
method:
> > > > > >
> > > > > > def amethod(self):
> > > > > > self.mydict = {}
> > > > > > self.mydict['a'] = 1
> > > > > >
> > > > > > (where self is the persistent object that is usually the
external
> > > > method's
> > > > > > "container")
> > > > > >
> > > > > > It wouldn't work as you expected. Although you'd see an 'a'
in
> > mydict
> > > > for a
> > > > > > little while in further accesses to it, 'mydict' would
eventaully
> > show
> > > > up as
> > > > > > an empty dictionary on the first access of it after it was
expired
> > from
> > > > the
> > > > > > RAM cache (after it was 'ghosted'), because the last thing
that the
> > ZODB
> > > > > > "saw" (via the __setattr__ on 'self' and a subsequent
transaction)
> > was
> > > > you
> > > > > > setting a empty dictionary.
> > > > > >
> > > > > > Persistent objects (like "self" in the above example) are
only smart
> > > > enough
> > > > > > to notice changes to themselves that happen through their
> > __setattr__
> > > > (e.g.
> > > > > > self.mydict = {} calls self's __setattr__). Mutating the
attribute
> > > > 'mydict'
> > > > > > above "in-place" (via self.mydict['a'] = 1) does not trigger
self's
> > > > > > __setattr__, so the ZODB never notices that "mydict" got
changed.
> > > > > >
> > > > > > There are two ways to handle this. The first is to treat
mutable
> > > > attributes
> > > > > > "immutably" via assigning to a temporary variable and then
making
> > sure
> > > > the
> > > > > > persistent container's __setattr__ gets called:
> > > > > >
> > > > > > def amethod(self):
> > > > > > dict = {}
> > > > > > dict['a'] = 1
> > > > > > self.mydict = dict # trigger persistence mechanism
implicitly
> > > > > >
> > > > > > The second is to use the _p_changed attribute of the
persistent
> > object
> > > > on
> > > > > > which the primitive is set. This explcitly tells the
persistence
> > system
> > > > to
> > > > > > include the object on which it's set into the current
transaction:
> > > > > >
> > > > > > def amethod(self):
> > > > > > self.mydict = {}
> > > > > > self.mydict['a'] = 1
> > > > > > self._p_changed = 1 # trigger persistence mechanism
manually
> > > > > >
> > > > > > Variations on this theme extend to list methods too (e.g.
> > list.append,
> > > > > > list.pop, etc.)
> > > > > >
> > > > > > "Custom" classes can certainly persist, they just need to
mix in the
> > > > > > "Persistence.Persistent" class as a base class.
> > > > > >
> > > > > > As long as you obey this (slightly annoying, but necessary)
rule,
> > you
> > > > > > shouldn't need to use PersistentMapping (it's really just a
> > convenient
> > > > > > wrapper that does this "magic" for you) or make any other
wrappers
> > for
> > > > other
> > > > > > mutable objects.
> > > > > >
> > > > > > I don't know why you're using __setstate__, but ummmm.. I
won't go
> > > > there.
> > > > > > :-) I didn't look at your product, but I don't think I need
to to
> > > > answer
> > > > > > your question...
> > > > > >
> > > > > > > Hi I am trying to get a handle on how I should handle
peristence
> > in my
> > > > > > > python products. I have read all the ZODB documents and
all the
> > > > Product
> > > > > > > tutorials, which all led me to believe that 1) mutable
objects
> > such as
> > > > > > > lists and dictionaries are not persistent and that updates
to
> > these
> > > > will
> > > > > > > not be implicitly saved into the ZODB, and 2) that custom
classes
> > > > would
> > > > > > > certainly not persist. That all seemed fine, and I used
persitent
> > > > > > > mapping and __setstate__ to fill things back in where
necessary.
> > But
> > > > > > > then I decided to demonstrate that persitence does break
as
> > suggested.
> > > > > > >
> > > > > > > I used boring product, added a dicitonary and a custom
class that
> > > > > > > contains it's own dictionary.... let the user update name
: value
> > > > pairs
> > > > > > > for both, and print the contents through index.dtml
> > > > > > >
> > > > > > > The problem is that everything persists fine !!!! through
> > restarts,
> > > > > > > everything?
> > > > > > >
> > > > > > > Why does it work??? shouldn't it not?
> > > > > > >
> > > > > > > I have attached the modified boring product ....
boringplus .....
> > it's
> > > > > > > dead simple to follow if you have made products before.
> > > > > > >
> > > > > > > Any explanation would be really nice.
> > > > > > >
> > > > > > > regards
> > > > > > > Matt
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > >
> > >
> > > _______________________________________________
> > > Zope maillist - Zope@zope.org
> > > http://lists.zope.org/mailman/listinfo/zope
> > > ** No cross posts or HTML encoding! **
> > > (Related lists -
> > > http://lists.zope.org/mailman/listinfo/zope-announce
> > > http://lists.zope.org/mailman/listinfo/zope-dev )
> > >
> > >
> >
> >
> > _______________________________________________
> > Zope maillist - Zope@zope.org
> > http://lists.zope.org/mailman/listinfo/zope
> > ** No cross posts or HTML encoding! **
> > (Related lists -
> > http://lists.zope.org/mailman/listinfo/zope-announce
> > http://lists.zope.org/mailman/listinfo/zope-dev )
> >
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )