[Zope] need advice on mass data processing

[Dieter Maurer]

Allen Huang wrote at 2007-1-9 01:05 -0800:
> ...
>I've never thought of using a BTree because I don't know enough about it. I'll look into it but will BTree still be a better choice than making instance if I'm going to make interactive functions?

BTrees provide an efficient means to store mappings

       key --> anything

In another message, I strongly recommended that you use
a "BTreeFolder2" (rather than a standard "Folder") to contain
your mass data. It uses internally a "BTree" (in fact, two of them)
as the name suggests.

If you have huge amounts of simple data, then it can be wise
to store the bare data and wrap it with an intelligent class
only when accessed. In this case, the most efficient way
woult be that "anything" above were a tuple of just your data.
Instead of a raw BTree, you would use your own container type
that uses a "BTree" to store the mass data but wraps the data
with your intelligent wrapper class on access.

You can look at "BTreeFolder2" on how this may work.
It does not wrap the data in a new (intelligent) class (which is
trivial)
but it wraps it in the acquisition context.



-- 
Dieter