[Zope-dev] Storing lots of big objects in containers

[Morten W. Petersen]

Hi guys,

A product I'm currently developing has a rather big (in many
ways) problem.  A product instance may store many, (thousands)
of big objects, ranging from 1-50 MB in size.

Now, whenever the product instance is accessed, with say for
example 100 objects with a size of 2MB each, Zope slows down
to a crawl, and spends a lot of time loading (I assume) the
objects from the ZODB, even though only 10 objects are
actually accessed.

..I can't really use any external storages, as the reason for
bringing these objects into the ZODB was to structure them
and make then indexable by ZCatalog.  And there's probably
no time for ZPatterns.

Someone suggested using BTreeFolder, but the documentation
and examples for it are rather scarce.   Does anyone know
of products that make a good example use of BTreeFolder?

Would BTreeFolder work for this?  Are there better alternatives?

Thank you for your time,

Morten