[Zope] 450 hits/s with 40+ TPS ..:D : Thanks to the Community

[Ben Last (Zope)]

> Trevor Warren
> >> RamCacheManager : Null (Get no performance boost
> with it on, some gurus say with more ZPT complexity it
> may help. But i had a nasty surprise. With the ZPT
> cached, there is total serving of the objects from mem
> which causes stale variables to be served. All efforts
> spent in making complex random quieries are naught
> with this option.)
That may depend on the nature of your actual production environment.  If
your RamCache is caching each template irrespective of the parameters sent
to it, then yes, you'll get stale data.  However, if, in practise, a given
template with the same given parameters returns the same result, a RamCache
is appropriate.
My understanding (based on a quick look at the source) is that the
RAMCacheManager indexes cached objects based on the request that generates
them.  Thus repeated accesses with exactly the same request URL will return
the same object (subject to cache expiry).  That's what caching *is*!
You can adjust the cache max age (it's the max_age parameter on creation),
so if, for example, your data isn't update more often than every n seconds
but requests come more frequently, then set max_age to something less than n
(say, n/2) and see if that helps.

Anyway, one reason I'm responding here is that I've been comparing effects
of RamCache with use of AcceleratedHTTPCache to manage an Apache 2 cache.  I
pretty much get equivalent results, with Apache caching *slightly* faster.
Appears that either Zope uses RAM to cache stuff, or Linux uses it to cache
the disk or file page cache that Apache uses.  Anyone else done this sort of
testing?

> >> hits/sec: 460 hits/s
> >> TPS : 40+
> >> System usage : <50%
Are you able to tell us how this compares with other setups (and other
technologies) you've tried?

>  Please write in for any further clarifications. I
> will still pursue other tests with complex sql's and
> will be in touch.
As a general rule of thumb in Zope testing, I've found that the more
complexity I shift to the SQL and the less processing I do in Python, the
faster it all goes (as one might expect).  Of course, you lose a certain
amount of flexibility that way, and given the choice I'd rather write Python
than SQL :)

Glad to hear that you've found the source of your problems, Trevor.

Ben Last
Technical Director
SleepyDog
07966 136572
ben at sleepydog.net
ben at benlast.com