[ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

[Hanno Schlichting]

On Fri, May 6, 2011 at 2:22 PM, Pedro Ferreira
<jose.pedro.ferreira at cern.ch> wrote:
> That's hard to do for a project that is already 8 or 9 years old, as you
> can see in the attached file, we've got have many cases that fall
> outside your limits. I've noticed, for instance, that pages that involve
> the loading of >200 MaKaC.review.Abstract objects have an awful
> performance record (maybe because we then load for each object a handful
> of other referenced persitent objects).

I'd expect load times per persistent objects to vary between 0.1 to
10ms. Over a network connection while sometimes hitting the disk, I'd
expect to see an average of 1ms. If you get something awful like an
Oracle real application cluster, with virtualization, storage area
networks and different data centers involved, you are lucky to see
10ms.

If you don't have any of the data in a cache and load hundreds of
objects, you very quickly get into the range of one to multiple
seconds to load the data. If you need to load more than 1000 objects
from the database to render a page, your database schema "sucks" (tm)
;-)

> But isn't RelStorage supposed be slower than FileStorage/ZEO?

The benchmarks vary on this a little bit. But for read performance
they are basically the same. You can do the same tricks like using
SSD's or bigger OS disk caches to speed up both of them. RelStorage
has simpler (freely available) clustering solutions via the native
database and supports things like the memcached cache.

Hanno