[Zope] persistence and dictionaries

[Chris McDonough]

All pickleable Python primitive types (strings, dictionaries, lists, Nones,
integers, floats, longs, etc.) can live in the ZODB.  They can persist just
like instances that inherit from the Persistent class.

I think you read a little too much in to the fact that you need to "treat
mutable objects immutably" when working with them in the ZODB.  This
statement doesn't mean that these kinds of objects can't be saved in the
ZODB, it just means you need to treat them specially when putting them in
the database.

For instance, if you were doing this inside of an external method:

def amethod(self):
   self.mydict = {}
   self.mydict['a'] = 1

(where self is the persistent object that is usually the external method's
"container")

It wouldn't work as you expected.  Although you'd see an 'a' in mydict for a
little while in further accesses to it, 'mydict' would eventaully show up as
an empty dictionary on the first access of it after it was expired from the
RAM cache (after it was 'ghosted'), because the last thing that the ZODB
"saw" (via the __setattr__ on 'self' and a subsequent transaction) was you
setting a empty dictionary.

Persistent objects (like "self" in the above example) are only smart enough
to notice changes to themselves that happen through their __setattr__ (e.g.
self.mydict = {} calls self's __setattr__).  Mutating the attribute 'mydict'
above "in-place" (via self.mydict['a'] = 1) does not trigger self's
__setattr__, so the ZODB never notices that "mydict" got changed.

There are two ways to handle this.  The first is to treat mutable attributes
"immutably" via assigning to a temporary variable and then making sure the
persistent container's __setattr__ gets called:

def amethod(self):
   dict = {}
   dict['a'] = 1
   self.mydict = dict # trigger persistence mechanism implicitly

The second is to use the _p_changed attribute of the persistent object on
which the primitive is set.  This explcitly tells the persistence system to
include the object on which it's set into the current transaction:

def amethod(self):
   self.mydict = {}
   self.mydict['a'] = 1
   self._p_changed = 1 # trigger persistence mechanism manually

Variations on this theme extend to list methods too (e.g. list.append,
list.pop, etc.)

"Custom" classes can certainly persist, they just need to mix in the
"Persistence.Persistent" class as a base class.

As long as you obey this (slightly annoying, but necessary) rule, you
shouldn't need to use PersistentMapping (it's really just a convenient
wrapper that does this "magic" for you) or make any other wrappers for other
mutable objects.

I don't know why you're using __setstate__, but ummmm.. I won't go there.
:-)  I didn't look at your product, but I don't think I need to to answer
your question...

> Hi I am trying to get a handle on how I should handle peristence in my
> python products.  I have read all the ZODB documents and all the Product
> tutorials, which all led me to believe that 1) mutable objects such as
> lists and dictionaries are not persistent and that updates to these will
> not be implicitly saved into the ZODB, and 2) that custom classes would
> certainly not persist.  That all seemed fine, and I used persitent
> mapping and __setstate__ to fill things back in where necessary.  But
> then I decided to demonstrate that persitence does break as suggested.
>
> I used boring product, added a dicitonary and a custom class that
> contains it's own dictionary.... let the user update name : value pairs
> for both, and print the contents through index.dtml
>
> The problem is that everything persists fine !!!!  through restarts,
> everything?
>
> Why does it work???  shouldn't it not?
>
> I have attached the modified boring product .... boringplus ..... it's
> dead simple to follow if you have made products before.
>
> Any explanation would be really nice.
>
> regards
> Matt
>
>