[Zope] Scaling problems, or something else?

[Paul Winkler]

On Tue, May 15, 2007 at 12:00:52PM +0200, Gaute Amundsen wrote:
> I set cache-size to 10000 last night, up from the default.
> I felt I had to try that before I had a grap in place, so I don't have good 
> numbers to estimate tha change, but with about 20 hits a sec on apache, there 
> was close to 500000 loads the last hour, and just 3000 writes. 
> Does that look reasonable?

Not really. Too many loads; you're getting a lot of cache misses and
blowing out the cache a lot. Keep on doubling the cache size until
your whole working set is in RAM all the time.

> The built in help seems to indicate I should increase the cache until reads 
> approaches zero..?

Yeah, for some definition of "approaches".  I think a more realistic
minimum would be number of threads * number of ZODB writes (since each
write potentially invalidates one cached object per thread).

It's hard to quantify the point at which you're caching enough, but
you're not even close.

> > Also, I don't think you've mentioned what sort of app this is.
> > Is it mostly reads or are there lots of writes?
> > "Mostly reads" is a lot easier to optimize :)
> >
> Big CMS system with about 70 virtual domains.

Based on your activity graph, you do indeed have a lot more reads than
writes.

What's the CMS based on? Plone? Silva?
Chris W. had a good point about the catalog.

Do you have blobby data in the ZODB? (large images or files)?  Those
tend to play havoc with zodb cache activity, since one OFS.Image is
stored as an arbitrarily long chain of small persistent objects.  So
whereas a Plone Document or a Page Template needs only one entry in
the cache, an Image might need hundreds.

--

Paul Winkler
http://www.slinkp.com