[Zope-dev] Help: __getstate__ overriding

[Tim Peters]

[Syver Enstad]
> I was aware that __setstate__ doesn't allow me to commit my changes after
> only loading the object into memory (__setstate__ is called). I may have
> explained myself unclearly (not a native english speaker/writer),

I don't think that matters much:  English instead of code is always
ambiguous, even for native writers.

> the problem is that it won't save my changes when I add more items to the
> _oidsToArticles BTree long after __setstate__ time.

Right.  I tried to explain that in my first reply.  It's expected.  You have
to get _p_changed set on your *object*.  It doesn't matter how much you
mutate new objects attached *to* the object you loaded, you have to get
_p_changed set to 1 on the original object.

> Example code:
>
> # articleDb is an ArticleDb and has the __setstate__ method
> articleDb = connection.root()['articledb']

At this point articleDb._p_changed is not 1, and nothing you do later
changes this fact.

> for each in articleDb.articles() # loop through all article
>     print each

It's obvious that this doesn't set articleDb._p_changed, right?

> computer = Computer(1030)
> articleDb.addArticle(computer)

That adds something to the BTree that is bound to an attribute of articleDb,
but still doesn't set _p_changed on articleDb (mutating the BTree does not
mutate the object containing the BTree).  So far as the *database* is
concerned, there's still no path from the root object to the BTree your
__setstate__() created, so the BTree never gets committed.  If you do

articleDb._p_changed = 1

at this point too, then your changes will get stored.

> connection.commit() # nothing is saved

Right.  As far as the persistence machinery is concerned, articleDb itself
never changed, so there was no reason to store it.  Since it didn't get
stored, the BTree you attached to it in __setstate__ doesn't get stored
either.

> Everything works smoothly until connection.commit(). The __setstate__
> method correctly converts to using the BTree and the addArticle method
> adds a new computer to ArticleDb.

The problem is that, while adding the BTree to articleDb did change (did set
_p_changed to 1) articleDb, because that happened as a side effect of
loading the object, the unghostification machinery clears _p_changed after
__setstate__ returns.  Then nothing you did after that set
articleDb._p_changed to a true value.

> If I instead create a new ArticleDb instance with the BTree instead of
> using __setstate__ the new computer is saved as it should.

Yes.  If you had called __setstate__ *directly*, it would also work
(contradicting the current minimal docs, but so it goes).  The problem is
that unghostifying "by magic" clears _p_changed.

> In none of the cases is articleDb._p_changed == True,

When you create an object directly, it's not associated with a Connection (a
"jar"), and some of the persistence machinery is sidestepped then.  New
objects only get into the database if you attach them to an object that's
already in the database, and the latter object is marked changed.  Think
about that <wink>, and the purpose of the root() object will become clearer.

> but when the ArticleDb is created "cleanly" instead of upgraded, commit
> works anyway.

Sorry, I'm not clear on what that means.  In any case, it doesn't really
matter how an object gets created, what matters is whether the persistence
machinery "sees" a path to it from a changed object already in the database.

> Since this works if the object that holds _oidsToArticles is created from
> scratch, it seems that ZODB isn't properly aware of the _oidsToArticles
> attribute since it was created in __setstate__ instead of in __init__.

Both ways set _p_changed *at the time* oidsToArticles is bound to the new
BTree.  The difference is that uhghostification deliberately clears
_p_changed after calling __setstate__.

> I would really like to know exactly what happens here as the reason I am
> utilizing an object database is to be able to refactor the database with
> a minimum of fuss, and I would expect __setstate__ to allow me to make
> additional attributes in my instances.

Calling __setstate__ as a side effect of a persistent load does not, by
itself, support making changes that persist.  It apparently wasn't designed
to, either (see the other msgs in this thread, particularly from ChrisM).