It's helpful to post your responses to the mailing list, that way when someone else has a similar problem in the future they'll be able to find the information. Inheriting from Persistent is also necessary to control the granularity of the database. Persistent objects are saved as separate `records` by ZODB. Other objects do not have a _p_oid attribute and have to be saved as part of their parent record. Laurence 2008/6/19 <AFoglia@princeton.com>:

Laurence Rowe wrote:

...

AFoglia@princeton.com wrote: Does your record class inherit from persistent.Persistent? 650k integers + object pointers should only be of the order 10 Mb or so. It sounds to me like the record data is being stored in the btrees bucket directly.

No, it does not. It's just a simple dictionary for the time being. I assumed the BTree bucket would itself know to load the values only when they are explicitly requested, and that the Persistence of the objects just merely meant that the database didn't keep track of changes of nonpersistent objects. I will try copying my dictionaries to Persistent Mappings for now.

...

Something like this should lead to smaller bucket objects where the record data is only loaded when you access the values of the btree:

...

...

...

from BTrees.IOBTree import IOBTree bt = IOBTree() from persistent import Persistent class Record(Persistent): ... def __init__(self, data): ... super(Record, self).__init__() ... self.data = data ... rec = Record("my really long string data") bt[1] = rec