[Zope] Re: Zope Digest, Vol 6, Issue 37

[nwingfield at che-llp.com]

Alessio,

We are piloting a very similar system (plain Zope, no Plone).  A few
lessons we learned that you might benefit from:

1.  Use DirectoryStorage for your ZODB.  It is highly stable, scalable,
resilient to corruption, easily backed up, etc, etc.  The secret to its
success is that it uses one file per Zope revision, rather than a single
Data.fs.  If you do so, you might also consider using reiserfs because it
is better with large numbers of files.  And there will be a lot!  (I
believe this only applies to Linux.)
2.  Do not store binaries in the ZODB.  Your ZODB will grow plenty large
with metadata revisions without shoving files into it, too.  Ours hit 20GB
at one point because I forgot to pack it for a while.  If at all possible,
store the files themselves outside of Zope.
3.  Use BTreeFolders if you can, to speed retrieval.  This may not be an
issue with Plone.
4.  Investigate ZCatalogs thoroughly to make sure they meet your needs.  We
are about to jump ship in favor of a relational database, simply because
ZCatalogs are not as well suited to our purposes.

Nathaniel

> I'm developing an intranet document management system with Zope/Plone
> and I'm concerned about the capabilities of ZODB. The system is going to
> store many thousands of large files, so the Data.fs is very likely to
> grow up to tens of GB. I'm wondering if ZODB can handle such amount of
> data in its standard implementation (using Data.fs). I investigated
> several alternatives, including ApeLib/Dbtab, but they don't seem to
> work out of the box with Plone (while thay do in ZMI).
>
> Is somebody aware of **fully-tested, real-world** applications with
> multi-GB storage or, in other words, should I stay with ZODB or search
> for alternatives?
>
> Thanks
>
> Alessio Beltrame