[Zope] Alternate storage for ZODB

[Toby Dickenson]

On Thursday 04 Jul 2002 3:57 pm, Thierry Florac wrote:

> I don't want to execute queries on Zope's database, but what I'd like t=
o
> be able to do is to use an external storage, which could provide
> advanced functions such as :

It looks like your concerns are the same as what prompted me to start on=20
DirectoryStorage.....


>  - replication (also available with ZEO),

ZEO doesnt do replication today. It is planned, but for far in the future=
=2E

>  - partitionning,

of what, and why?

>  - handling of very high volumes,

As in file size, or throughput?

FileStorage leads the throughput benchmarks, but even the slowest storage=
 is=20
not sufficiently slow to make a difference (IMO).

All of today's production storages have serious problems when packing=20
databases with many objects, because they all use an in-memory index of=20
reachable objects. For high volume you need a non-versioning storage (or=20
DirectoryStorage, when its ready)

> In fact, I'm a little bit afraid by the ZODB FileStorage, and by it's
> potential ability to handle very big data files correctly... We have a
> lot of big files to publish and index, and our ZODB will probably have =
a
> size of 10 GB quite rapidly... Any advise ??

Ive never had a filestorage that big. If memory usage is linear (which I =
think=20
is a reasonable assumption) then a FileStorage that big will need (roughl=
y)=20
1G of ram for normal use, and 2-3G during packing. If thats not a problem=
; do=20
it.