Hi,
[stuff deleted where I describe that my zope memory use seems to grow unreasonably (at least, that's what I think) with lots of BTrees._IOBTree.IOBucket objects] Many of them are likely "ZCatalog" metadata blocks. They can become huge monsters.
Unless you do statistics or present huge numbers of hits in a single page, you do not need metadata. Access the object instead (via "proxy.getObject()").
Under no circumstances put large fields in the metadata table.
Also remove the "bobobase_modification_time" from this table. Tweaking the catalog use and what's in it helps marginally. Thanks for the tips. However I still have memory problems. For example:
I've packed the database - it is now ~300 MByte. If I try to export the main object, i.e. the database containing all our travel information, zope runs out of memory, gobbling up the 512 Mbyte RAM I have per process in a few minutes. I then cleared the catalog and now the zexp sometimes (but not always) makes it. Also, I reduced the number of threads and the number of objects per thread, but this doesn't seem to help. Douwe Osinga