[ZODB-Dev] Very large amounts of output from fsrefs.py

[Benji York]

On Wed, Aug 27, 2008 at 7:32 AM, Ingvar Hagelund <ingvar at linpro.no> wrote:

> Now, writing to the ZEO database is very slow

Hard to tell the cause of that from here.  Strace, tcpdump, and
wireshark are my friends.

> and the same goes for
> searching, which is a problem for the users that put information into
> the solution.

Larger ZODB object caches and ZEO disk caches might help there.

> Average cpu usage on the box running ZEO is some 10%, and never above
> 65% (of one cpu core).

Watching the output "iostat -x 60" (especially the % utilization) can be
helpful in identifying disk bottlenecks (which I'm not especially
suspicious of in this case).

> The size of Data.fs is quite large, about 8.2GB after packing.

I wouldn't worry about a Data.fs of that size.  They can comfortably
grow much larger than that.

> Now for some questions:
>
> - Is a configuration like the one described a sensible setup?

Yes.

> - Are there any ovious key tuning buttons that we have missed?

See the message from Marius for the most likely culprit.

> - Would it be an idea to move binaries out of Data.fs and serve them
>  from a shared filesystem instead?

Maybe, maybe not; the details are important there.  My suspicion is that
you wouldn't see much benefit.
-- 
Benji York
Senior Software Engineer
Zope Corporation