[Zope-DB] Restriction of ZOPEDB

[Dieter Maurer]

Robert Sösemann wrote at 2005-3-17 17:09 +0100:
>in one of our projects we are using ZOPEDB for storage of documents and binary files? On the ZOPE Server the project is running on, we have mayn other ZOPE apps running. As the ZOPEDB of the project has reached a enormous size (1 GB) we are facing performance issues. This is affecting also all the other ZOPE apps on this server.

Usually, performance it not affected by the storage size.
However, you need RAM proportional to the number of objects
in the storage (to maintain a map from object id to file position).

> ...
>The important questions are:
>----------------------------
>- What are the limits/restrictions of the ZOPEDB due to size and file amount?

There are virtually none.

We are using ZODB storages in the order of 20 GB.

>- Besides performance issues, which other problems could we get? Especially side effect that affect the other apps on this server).

RAM requirements increase linearly with the number of objects
in a storage.

>- Could we switch to using a binary storage outside of the ZOPEDB? Doe you have any experience how much effort that is for a running system?

There is "DirectoryStorage" and several products that
store large binary objects in the file system (e.g.
"ExtFile", "ExternalFile", ...).

Formerly, we used "ExtFile". Nowadays (where all modern
file systems support files larger than 4 GB and "repozo" supports
incremental backup for FileStorage), we no longer bother
with the additional complexity...

>- Or is it just more efficient to put more "power" (CPU, RAM, HD) on the server?

You should analyse your application with a profiler
and find out what really is the limiting factor.
I can recommend (my) ZopeProfiler:

  <http://www.dieter.handshake.de/pyprojects/zope>


-- 
Dieter