Am Dienstag, 9. August 2005 02:03 schrieb Andrew Langmead:
(on these machines, the ZEO client cache is set to 2GB and a cache flip occurs maybe twice a week.)
I think this part to be quite interesting. We too are experimenting with the optimal cache sizes, where our (daily) packed Data.fs is "only" about 7 GB. Its only since friday that we updated to the latest 2.8-svn codebase, to get the latest fixes of the ZODB/ZEO cache tracing parts. Our setup is one ZEO server, and two clients on seperate machines. A third client is on the same machine as the server, which is only used for admin/maintenance tasks. What we've seen so far: we used to set the cache-size to 250 MB; and every know and than we have to observe that after a restart of a client a cache verification is necessary (while not always obvious, why this happened), which often takes "forever" (that is, 10 to 20 minutes are possible). And that during such a verification, the whole system seems to be down (in the sense that delivery of pages is slow as hell :-(). I think that in such a case simply deleting the cache file and starting with a fresh one is way faster, but have no hard numbers to prove that feeling. Now we look what happens at different cache sizes. While it might be too early to really make conclusions, at least it might give some hints: We have set the cache on one machine to be 200 MB, and on the other to 100 MB. Up to now, even the 100 MB cache gives a hitrate of better than 98 % (!) most of the time. What I hope is, that the smaller the cache size, the faster the verification, if necessary. Again, no real hard numbers to check this (seems to be hard to get some, sometimes the log says things like 2005-08-08T00:10:55 INFO ZEO.ClientStorage (27891) No verification necessary (last_inval_tid up-to-date) ------ 2005-08-08T00:10:55 INFO ZEO.ClientStorage (27891) Waiting for cache verification to finish ------ 2005-08-08T00:10:55 INFO ZEO.ClientStorage (27891) Verifying cache ------ 2005-08-08T00:10:55 INFO ZEO.ClientStorage (27891) endVerify finishing ------ 2005-08-08T00:10:55 INFO ZEO.ClientStorage (27891) endVerify finished huh? no verifictation necesarry, but waiting for verification to finish? sometimes, the "endVerify" messages never appear; and you never know which of our three storages the messages are about...). We increased the servers invalidation-queue-size from 100 to 1000 along the way, don't know yet if this has any effect. Having said all this, our scenario might not be common, as a major part of the traffic concentrates on a small part of the site, which might be the reason that even a small cache seems to be sufficient here. I'm wondering if anybody else has seen long lasting cache verifications, which seems to block the whole system. Or if someone knows how to prevent this from happening at all... Cheers, Sascha