If we have a cmf based site with over 500 pages that are CMFDefault.Documents and we are using ZEO with 3 Zope Clients because we have 3 servers in a server farm behind a content switch that does loadbalancing, We're trying to understand all our caching options so we can avoid all 3 clients always hitting the ZEO Server since that sort of defeats the purpose of it being load balanced. Almost of the 500 pages are of a nature that they can be cached since they are not dynamic in the "personalization" sense. (We aren't using the CMF for it's portal like (member join) abilities ... but rather for it's separation of content and workflow abilities for a limited number of employees to update content. Aside from the limited number of employees we set up as owners w/ local roles for maintaining stuff, those hitting our site are always "Anonymous", and the info isn't personalized.) We're thinking about going with mod_proxy's cache abilities because we're using Stronghold which isn't up to Apache 2 yet to use mod_cache. But where I'm confused is about the layer's of caching available in zope. I've tried to read up .. but can't quite tell when it's advised nor how to make use of zope's caching abilities? For instance in a client .. under database management, cache parameters I see this (which I've never messed with so it must be by default): Total number of objects in the database 20518 Total number of objects in all caches 2100 Target Size 400 Target max time between accesses 60 Can someone point me to understanding if this level of caching stops the client from making the trip to the ZODB on the ZEO Server. Does is make the mod_proxy way less necessary? How are they different? What else should I know to make these decisions? Do the needs for caching of a CMF site differ from a non CMF site, since every CMF content object is having to have a skin applied when it's requested? Thanks, Lynn
On Mon, 21 Jan 2002 12:20:41 -0600, Lynn Walton <waltonl@franklin.edu> wrote:
For instance in a client .. under database management, cache parameters I see this (which I've never messed with so it must be by default):
Total number of objects in the database 20518 Total number of objects in all caches 2100 Target Size 400 Target max time between accesses 60
Can someone point me to understanding if this level of caching stops the client from making the trip to the ZODB on the ZEO Server.
That stops Zope needing to talk to its storage (FileStorage or ZEO ClientStorage), then unpickle the result, for every object access. The http client is not involved in this level. ps interesting numbers. How many publisher threads do you have? (thats -T on the command line) Toby Dickenson tdickenson@geminidataloggers.com
On Tuesday 22 January 2002 02:50 am, Toby Dickenson wrote:
On Mon, 21 Jan 2002 12:20:41 -0600, Lynn Walton <waltonl@franklin.edu>
wrote:
For instance in a client .. under database management, cache parameters I see this (which I've never messed with so it must be by default):
Total number of objects in the database 20518 Total number of objects in all caches 2100 Target Size 400 Target max time between accesses 60
Can someone point me to understanding if this level of caching stops the client from making the trip to the ZODB on the ZEO Server.
That stops Zope needing to talk to its storage (FileStorage or ZEO ClientStorage), then unpickle the result, for every object access.
The http client is not involved in this level.
ps interesting numbers. How many publisher threads do you have? (thats -T on the command line)
what's interesting? when i looked at thenumbers, it's like staring at the wall :P could you please enlighten this poor soul? thanks
Toby Dickenson tdickenson@geminidataloggers.com
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Tuesday 22 January 2002 9:13 am, bak (kedai) wrote:
On Tuesday 22 January 2002 02:50 am, Toby Dickenson wrote:
On Mon, 21 Jan 2002 12:20:41 -0600, Lynn Walton <waltonl@franklin.edu>
Total number of objects in the database 20518 Total number of objects in all caches 2100 Target Size 400 Target max time between accesses 60
Can someone point me to understanding if this level of caching stops the client from making the trip to the ZODB on the ZEO Server.
That stops Zope needing to talk to its storage (FileStorage or ZEO ClientStorage), then unpickle the result, for every object access.
The http client is not involved in this level.
ps interesting numbers. How many publisher threads do you have? (thats -T on the command line)
what's interesting? when i looked at thenumbers, it's like staring at the wall :P
The target size of 400 applies to each ZODB object cache. Each zope instance has more than one cache; generally its one per publisher thread (-T), plus a few more if you are using Sessions. The default -T value is 4. That is probably more than enough to support 3 clients, so I guess Zope is only using 2 of those threads (exceptions would be if some of those clients are a high-concurrency system stress tool, or if the site has some custom methods that take a long time to render) If Lynn is using that default and no sessions then each cache contains somewhere between 2100/4 = 525 objects (if all threads/caches are used) and 2100/2 = 1050 objects (following my guess that only 2 threads/caches are used). Either way, that is quite a bit more than the target of 400. The ZODB cache does behave that way, but only under reasonable memory pressure. I define memory pressure as objects touched per transaction (or http request). I am suprised that CMF involves sufficient memory pressure. I am currently looking at redesigning the ZODB cache to have a stiffer response to memory pressures.... hence my interest.
Thanks to bak (kedai) for asking the question that led to this explanation as I definitely needed the additional info. Thanks Toby for explaining further. Actually, we haven't moved to the 3 zope client set up yet so the numbers I showed are for 1 zope client using the ZEO Server. We didn't pass any -T parameters so I guess we'd have the default of 4. (I searched for info on publisher threads and came up empty .... so I wouldn't have known the default was 4.) I don't know if there is any relation between your guess that so many objects in cache is a sign of memory stress, and the fact that we are experiencing Zope crashes 1-3 times per day (always associated with our higher load times). We have been working for many weeks with ZC support to find the cause and so far nothing we've tried has made any difference. (compiling without pymalloc, restrictedPython patch, etc) Our load on that site is probably around 70000 requests per day but we have our images being served up by Apache rather than in zope and so if you subtract the requests for images, I estimate no more than 20000 requests per day that zope is handling. Most of those are CMF Documents that get rendered with a zpt template that does make an external method call but not one that is "highly intensive". In fact hardly any zope requests at this point are to anything "intensive". If you have anymore insights based on the info I've now shared, please let me know. I appreciate it. Lynn Toby Dickenson wrote:
On Tuesday 22 January 2002 9:13 am, bak (kedai) wrote:
On Tuesday 22 January 2002 02:50 am, Toby Dickenson wrote:
On Mon, 21 Jan 2002 12:20:41 -0600, Lynn Walton <waltonl@franklin.edu>
Total number of objects in the database 20518 Total number of objects in all caches 2100 Target Size 400 Target max time between accesses 60
Can someone point me to understanding if this level of caching stops the client from making the trip to the ZODB on the ZEO Server.
That stops Zope needing to talk to its storage (FileStorage or ZEO ClientStorage), then unpickle the result, for every object access.
The http client is not involved in this level.
ps interesting numbers. How many publisher threads do you have? (thats -T on the command line)
what's interesting? when i looked at thenumbers, it's like staring at the
wall :P
The target size of 400 applies to each ZODB object cache. Each zope instance has more than one cache; generally its one per publisher thread (-T), plus a few more if you are using Sessions.
The default -T value is 4. That is probably more than enough to support 3 clients, so I guess Zope is only using 2 of those threads (exceptions would
be if some of those clients are a high-concurrency system stress tool, or if the site has some custom methods that take a long time to render)
If Lynn is using that default and no sessions then each cache contains somewhere between 2100/4 = 525 objects (if all threads/caches are used) and
2100/2 = 1050 objects (following my guess that only 2 threads/caches are used).
Either way, that is quite a bit more than the target of 400. The ZODB cache
does behave that way, but only under reasonable memory pressure. I define memory pressure as objects touched per transaction (or http request). I am suprised that CMF involves sufficient memory pressure.
I am currently looking at redesigning the ZODB cache to have a stiffer response to memory pressures.... hence my interest.
On Tuesday 22 January 2002 5:53 pm, you wrote:
20000 requests per day
Aha! your original post mentioned three clients, and I took that to mean end-user clients (rather than Zope clients to your ZEO server). Your memory usage statistics are quite believeable for that size of load. Sorry for the distraction.
I don't know if there is any relation between your guess that so many objects in cache is a sign of memory stress, and the fact that we are experiencing Zope crashes 1-3 times per day (always associated with our higher load times). We have been working for many weeks with ZC support to find the cause and so far nothing we've tried has made any difference. (compiling without pymalloc, restrictedPython patch, etc)
I think its unlikely to be a cause, but if you are desperate then any data point can be useful..... It may be interesting to see if those numbers change during your uptime lifecycle. It is possible to get unusually big numbers right after the server starts, but it should settle down to an equilibrium. If the numbers seem to be growing then this may be a clue to a problem. If this is the case, I suggest you have a look at the reference counts section of the debugging control panel tab for further clues. Toby Dickenson tdickenson@geminidataloggers.com
participants (4)
-
bak -
Lynn Walton -
Toby Dickenson -
Toby Dickenson