[ZODB-Dev] Very large amounts of output from fsrefs.py

[Paul Winkler]

On Wed, Aug 27, 2008 at 11:03:54AM -0400, Marius Gedminas wrote:
> On Wed, Aug 27, 2008 at 01:32:53PM +0200, Ingvar Hagelund wrote:
> > We have an solution based on a third party zope based framework, with 12
> > Zope instances accessing ZODB via ZEO.
> ...
> > Now, writing to the ZEO database is very slow, and the same goes for
> > searching, which is a problem for the users that put information into
> > the solution.
> ...
> > We want to sort out if there are any obvious data points on tuning ZEO
> > that we have missed, or that there are errors or corruption in the database.
> ...
> > Software environment: ZEO from Zope 2.10.3, python-2.4.4, Debian
> > GNU/Linux, Data.fs on ext3 filesystem
> 
> ZEO had a bug, fixed in rev 73871, where it would spend up to 10ms (your
> OSes scheduling timeslice) in idle sleep for every object received from
> the ZEO server, resulting in very long request processing times on an
> otherwise idle server, if a request had to load hundreds or thousands of
> persistent objects.

No kidding??  I bet that was a factor in the benchmark results I saw a
few years ago: loading an uncached, largish OFS.Image.File instance
would take an order of magnitude longer from ZEO on localhost compared
to FileStorage. See the end of http://slinkp.com/code/zopestuff/blobnotes/

I've just updated that page with new results at the bottom: ZEO
performance for OFS.Image.* classes is still a bit slower, but now
much closer to FileStorage speed.

The relevant fix appears to have made it into both the Zope 2.10 branch (see
r73967) and 2.11.


-- 

Paul Winkler
http://www.slinkp.com