On Fri, 2007-02-23 at 12:09 -0500, Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Tres Seaver wrote:
Roché Compaan wrote:
On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote:
Roché Compaan wrote:
I'm curious, has anybody played around with the idea of caching ZCatalog results and if I submitted a patch to do this would it be excepted?
I quickly coded some basic caching of results on a volatile attribute and I was really surprised with the amount of cache hits I got (especially with a Plone site that is a heavy user of the catalog) +1. I think using the 'ZCachable' stuff (e.g., adding a RAMCacheManager and associating a catalog to it) would be the sanest path here. Cool idea. I haven't done any coding involving OFS.Cache though. Looking at it briefly it looks like one can modify the catalog to subclass OFS.Cacheable and then use the ZCacheable_get, ZCacheable_set and ZCacheable_invalidate methods to interact with a cache manager. This needs to be pretty explicit though. Are there any side effects that I should guard against if the catalog subclasses OFS.Cache?
I don't think so. Here are some random thoughts on the idea:
- The 'searchResults' method must pass its keyword arguments as part of the cache key.
- I don't know if there is a reasonable way to do 'mtime' for the catalog: we would like to be able to get an mtime cheaply for the BTrees (indexes, the 'data' container), but I don't know if that is possible.
- The "right" place to do this feels like the 'searchResults' of ZCatalog, just before it calls 'self._catalog.searchResults'.
- The CMF's catalog overrides 'searchResults', but calls it at the end, so everything there should work.
In my prototype I also wired the caching into searchResults: def searchResults(self, REQUEST=None, used=None, _merge=1, **kw): ... cache_key = None if args: cache_key = self._makeCacheKey(args) result = self._getCachedResult(cache_key) if result: return result return self._cacheResult(cache_key, self.search(args, sort_index, reverse, sort_limit, _merge))
Hmm, on further thought:
- It isn't safe to stash persistent objects in the RAM Cache manager, because they can't be used safely from another database connection.
But the lazy map of brains isn't persistent?
- The result set you get back from a query is a "lazy", which will be consumed by each client: no two clients will see the same thing.
I don't follow. The Lazy will contain a set of document ids that will be the same for all clients, not? I got satisfactory results by storing results in a volatile attribute (and they are not shared by clients). I'm still curious to see what can be achieved with ZCacheable to extend the lifetime of the cache. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za