[Syver Enstad]
I was aware that __setstate__ doesn't allow me to commit my changes after only loading the object into memory (__setstate__ is called). I may have explained myself unclearly (not a native english speaker/writer),
I don't think that matters much: English instead of code is always ambiguous, even for native writers.
the problem is that it won't save my changes when I add more items to the _oidsToArticles BTree long after __setstate__ time.
Right. I tried to explain that in my first reply. It's expected. You have to get _p_changed set on your *object*. It doesn't matter how much you mutate new objects attached *to* the object you loaded, you have to get _p_changed set to 1 on the original object.
Example code:
# articleDb is an ArticleDb and has the __setstate__ method articleDb = connection.root()['articledb']
At this point articleDb._p_changed is not 1, and nothing you do later changes this fact.
for each in articleDb.articles() # loop through all article print each
It's obvious that this doesn't set articleDb._p_changed, right?
computer = Computer(1030) articleDb.addArticle(computer)
That adds something to the BTree that is bound to an attribute of articleDb, but still doesn't set _p_changed on articleDb (mutating the BTree does not mutate the object containing the BTree). So far as the *database* is concerned, there's still no path from the root object to the BTree your __setstate__() created, so the BTree never gets committed. If you do articleDb._p_changed = 1 at this point too, then your changes will get stored.
connection.commit() # nothing is saved
Right. As far as the persistence machinery is concerned, articleDb itself never changed, so there was no reason to store it. Since it didn't get stored, the BTree you attached to it in __setstate__ doesn't get stored either.
Everything works smoothly until connection.commit(). The __setstate__ method correctly converts to using the BTree and the addArticle method adds a new computer to ArticleDb.
The problem is that, while adding the BTree to articleDb did change (did set _p_changed to 1) articleDb, because that happened as a side effect of loading the object, the unghostification machinery clears _p_changed after __setstate__ returns. Then nothing you did after that set articleDb._p_changed to a true value.
If I instead create a new ArticleDb instance with the BTree instead of using __setstate__ the new computer is saved as it should.
Yes. If you had called __setstate__ *directly*, it would also work (contradicting the current minimal docs, but so it goes). The problem is that unghostifying "by magic" clears _p_changed.
In none of the cases is articleDb._p_changed == True,
When you create an object directly, it's not associated with a Connection (a "jar"), and some of the persistence machinery is sidestepped then. New objects only get into the database if you attach them to an object that's already in the database, and the latter object is marked changed. Think about that <wink>, and the purpose of the root() object will become clearer.
but when the ArticleDb is created "cleanly" instead of upgraded, commit works anyway.
Sorry, I'm not clear on what that means. In any case, it doesn't really matter how an object gets created, what matters is whether the persistence machinery "sees" a path to it from a changed object already in the database.
Since this works if the object that holds _oidsToArticles is created from scratch, it seems that ZODB isn't properly aware of the _oidsToArticles attribute since it was created in __setstate__ instead of in __init__.
Both ways set _p_changed *at the time* oidsToArticles is bound to the new BTree. The difference is that uhghostification deliberately clears _p_changed after calling __setstate__.
I would really like to know exactly what happens here as the reason I am utilizing an object database is to be able to refactor the database with a minimum of fuss, and I would expect __setstate__ to allow me to make additional attributes in my instances.
Calling __setstate__ as a side effect of a persistent load does not, by itself, support making changes that persist. It apparently wasn't designed to, either (see the other msgs in this thread, particularly from ChrisM).