[ZODB-Dev] Re: ZODB Benchmarks

[Dieter Maurer]

Roché Compaan wrote at 2008-2-7 21:44 +0200:
> ...
>There are use cases where having a container in the ZODB that can handle
>large volumes and maintain a high insertion rate would be very
>convenient. An example of such a use case would be a site with millions
>of members where each member has their own folder containing different
>content types. The rate at which new members register is very high as
>well

I do not believe that the insertion rate the ZODB can handle now
would not be sufficient to handle this use case.

I do not have your timings present, but from our installation
I know that the ZODB can handle 10 transactions per second.
This would mean about 36.000 per day (10 hour days)
and about 1 million in a month.

>so the folder needs to handle insertions quickly. In this use case
>you are not dealing with structured data. If members in a site with such
>large volumes start to generate content, indexes in the ZODB become
>problematic too because of the slow rate of insertion.

We have several write intensive applications with storages
in the order of 10 to 20 GB and 10 to 20 millions objects
-- and have not yet seen problems with the insertion rate.

We do see other problems (notably commit contention)
but these problems cannot be solved by an increased insertion
rate.

>And it this point
>you start to stuff everything in relational database and the whole
>experience becomes painful ...

We speak again when you observe a concrete problem in a real
installation caused by limited insertion rate.



-- 
Dieter