[ZODB-Dev] Space used by IOBTrees

[Martijn Faassen]

Andreas Jung wrote:
> A XML document with about 26.000 nodes (1.3 MB data) is represented
> in an application by a nested structure of nodes where the childs
> are stored in IOBTrees (17.000 nodes out of the 26.000 nodes are leafs).
> The complete object allocates about 20MB inside the ZODB. analyze.py
> shows that there are about 9600 IOBTree objects inside the ZODB with an
> average size of 1850 bytes. The amount of data stored as attributes
> of a single treenode instance is very small (not more than 100-200 bytes).
> So why is the pickle of an IOBTree nearly 2KB large instead of several
> hundred bytes?

I was going to confirm a problem here, but then I updated my Zope3
checkout and recompiled the C extensions, and suddenly a lot of
incredible bloat that I was finding in my tests just evapored. Now my
XML documents in Forest including some indexes seem to get a bloat factor
of between 2 and 3 on my initial testing, which is quite tolerable.

Regards,

Martijn