[ZODB-Dev] fullstorage pack problem

[Tim Peters]

[Lorenzo Musizza]
> Hi all, I am using Zope 2.7.0, python 2.3.3 with a fullstorage bdb
> database, using the zope.conf defaults. The size is approx 900Mb, but I
> am unable to pack it manually (using ZMI), because it starts packing but
> never ends (more than 2 days on a 2Ghz 1G ram machine, then I cancelled),
> and the size reported by the zmi and the space on disk more than
> doubled...2100Mb. If I export all my sites, delete the fullstorage
> environment, recreate a new one and reimport I have again 900Mb, but the
> size grows over time and I can't manually pack it. I know it should do gc
> automatically but then why does it grow (since if I export and reimport
> the size is reduced)? Thanks in advance for any advice,

I'm sorry I can't hold out much hope for resolving this.  Zope Corp has
dropped the BDB-based storages for ZODB 3.3, because nobody (including
internally) has wanted them enough to do the work on them they need.

FullStorage doesn't do automatic gc, and I'm not sure where you got that
idea.  It requires packing, as does any storage with unlimited undo.  It did
have an "autopack" feature, though -- is that what you're thinking of?
autopack could clean up non-cyclic garbage, but you still needed "a real
pack" to get rid of unreferenced cycles.

That a manual pack didn't complete in two days is probably a bug, but since
FullStorage isn't supported by ZC anymore "the community" would need to
diagnose and fix it.  Alas, there doesn't appear to be a FullStorage
community to draw on, just isolated reports of people with problems <wink>.

The DB growth during packing is at least mysterious, but the underlying
Berkeley DB has mysterious growth behavior of it own, and it's unclear even
where to start looking on that one.  Best guess is that it's a shallow
consequence of the bug(s) preventing pack from completing.

Bottom line:  the practical advice at this time is to give up trying to use
FullStorage (or any other BDB-based storage).