On Thu, Aug 5, 2010 at 2:36 AM, Martin Aspeli <optilude+lists@gmail.com> wrote: ...
I have a package (plone.registry) that currently has a persistent structure like this:
Registry(Persistent) | +--> Records(Persistent) | +--> BTree of Record(Persistent) | +--> PersistentField(Persistent)
That is, a Registry is a persistent object containing a persistent Records object that in turn contains a BTree of persistent Record
Since BTrees are mapping, I assume that you mean the values are records and that the keys are something boring like strings or integers. I like to use mathematical notation when talking about BTrees and sets, as in: Registry BTree {? -> Record}
objects that contain a persistent PersistentField and a primitive value.
This is quite inefficient, though, because it results in a lot of object loads. On any given request, some of our projects load a dozen or more values from the registry. Each is just a simple primitive, but we need to load the full shebang to make it work.
Not sure what you mean by "full shebang".
Now, I'd like to move to this structure:
Registry(Persistent) | +--> Records | +--> BTree of Field | +--> BTree of values
I'm foggy on what "field" and "value" are here or what your queries are doing. Maybe this is just a distraction.
Here, there's only one Persistent object, plus the two BTrees: one holding all the fields and one holding all the values. Records no longer needs to be persistent (its attributes become part of the parent Registry's _p_jar).
I wonder what role "Records" plays independent of the "Registry". I also wonder why it matters whether it is persistent or not.
Fields no longer need to be persistent either, since they are in effect immutable objects. Values are primitives anyway.
I've done this (in a branch) and it works for new sites. However, I'm having a nightmare trying to migrate old sites. As soon as I access anything that uses the registry, I get ZODB errors, because the persistent structure is now different. In particular, it's trying to read a value into e.g. a Records object that used to derive from Persistent, but now no longer does.
What savings do you get by making Records non-persistent?
What is the best way to manage this type of migration?
Today, it probably makes the most sense to make new classes for the non-persistemnt objects. You'll then need to write a script to rebuild the data structures. Jim -- Jim Fulton