[Zope] Folder with one million Documents?

[Thomas Guettler]

Casey Duncan wrote:

>--- Joachim Werner <joe@iuveno-net.de> wrote:
>
>>Hi!
>>
>>Just my 2 eurocents:
>>
>>>I am developing a simple DMS. Up to now I use a python product with a
>>>
>>>BTreeFolder which
>>>contains all the documents. Every document gets an ID with
>>>
>>>DateTime().millis(). There will
>>>be up to 50 users working at the same time. And in the end I will have
>>>
>>>up to 3 million documents.
>>>
>>>Is there a better class than BTreeFolder for such mass storage?
>>>
>I thoroughly agree. Having developed a DMS myself, My
>cut-off point (which is really just an engineering
>intuition more than anything) was at about 5000
>documents, it would be best to store them directly in
>the file system.
>

Unfortunately my documents are not static files. They are python 
classes. They contain
fields for meta data and sometimes binary files.

Since my documents are not files I think it is not usefull to use a 
filesystem. The next
idea would be to have a database (maybe mysql, because it is fast) with one
table containing two colums: an ID and the pickled object.

Or you could use berkley DB for this.

>
>BTW: If you do set up any nifty FS storage solution, I
>would be interested in seeing it for future version of
>DocumentLibrary.
>
This list helped me  a lot. I will try to give it back and write a 
little howto if I
got a good solution.

 thomas