[Zope-dev] Request for comments: Directory storage

[Michel Pelletier]

Jimmie Houchin wrote:
> 
> From my naive understanding, would this help with the problem ZODB has
> with regard to folders with many objects? Would a person who is using
> DirectoryStorage not necessarily be required to partition their objects
> into an artificially derived hierarchical directory structure?
> 
> In other words can it be a possible solution to
> http://www.zope.org/Wikis/zope-dev/ReallyBigFolders ?

No.  The problem with ReallyBigFolders is that currently folders store
their children in a python dictionary.  When one object is accessed from
the folder, the entire dictionary is loaded into memory.  This problem
is independent of the storage.

I'm not sure how ReiserStorage works, but storages in general know
nothing about the containment relationships of the objects they store. 
The storage just considers them 'records' with a certain id.

The solution to making ReallyBigFolders is very similar to RFS though,
BTrees; except in the case of RFS records are stored as files which are
stored efficiently as btrees, and in the case of ReallyBigFolders
sub-objects are stored as nodes in BTrees (which eventually become
records which are stored as files which are stored as nodes in
BTrees...)
 
-- 

-Michel Pelletier

http://www.zope.org/Members/michel/MyWiki

Visit WikiCentral for the latest Zen:

http://www.zope.org/Members/WikiCentral