[Zope-dev] Re: ZOBD and pointers

[Tim Peters]

[Laurence Rowe]
> ...
> Unless you use a special PersistentList ZODB will have no choice but
> to store a new copy of the whole list when that list is modified.

Caution:  that's true of a PersistentList too.  The purpose of
PersistentList isn't realy to supply more-effecient storage (that's
the purpose of the various BTree classes).  The purpose of
PersistentList is this:

    myobject.my_list_attibute[3] = 4

If my_list_attribute is a plain Python list, the persistence machinery
has no way to know that my_list_attribute's state mutated, so the
assignment above will not get stored to disk at the next commit unless
you _also_ do

    myobject._p_changed = True # or 1

If my_list_attribute is a PersistentList, then the persistence
machinery does know when its state mutates, and there's no need to
manage _p_changed manually.

But in either case, the entire state of my_list_attribute gets stored
to disk whenever any part of it changes.  The only difference in what
gets stored in the example above is that myobject's state also gets
stored to disk if my_list_attribute is a Python list (assuming
myobject._p_changed gets set to a true value by hand), while
myobject's state does not need to get written to disk again if
my_list_attribute is a PersistentList (then myobject refers to
my_list_attribute via the latter's oid, and that oid hasn't changed,
so there's no need to store myobject's state again).  The entire state
of the list attribute gets written out in either case.

> If you have long lists then this can be a big problem.

Very true.

> The Persistent classes have special handling to make them more efficent.

Sometimes true, but not in the PersistentList case.

> So instead of lists use PersistentLists

If the goal is to save space, generally no, PersistentList won't help
that; to the contrary, their state takes a little more space on disk
than a plain list.

> and instead of dicts use BTrees,

That one's differenent:  a BTree is really a graph of (potentially
_very_) many distinct perisistent objects, and BTrees were designed to
support space- and time- efficient mutation.

> as these may be stored more efficiently in the ZODB.

For BTrees, yes.