[Zope] Caching Catalog results for performance

[Dieter Maurer]

Heimo Laukkanen wrote at 2004-2-26 00:12 +0200:
> ...
>Thanks for the tip. I haven't used module namespace before, though I read  
>in ZODB docs about possibility to use either volatile attributes or module  
>namespace. Are there - besides ZEO - any issues that I should be aware of  
>before doing it?

You must never put something in such variables that contains
references to persistent objects.

This means, you would not store the result of "searchResults"
but the raw result of "search" (a sequences of "rid"s)
and wrap them only when used (This is similar to the way
Cache Controlled Z SQL Methods
(http://www.dieter.handshake.de/pyprojects/zope) manage their cache).

>And is there actually anything else in it than creating a  
>module with functions that get and set data, and flush cached data.

You have 2 level structures.
You may get race conditions but this is (almost surely) not
an issue for a cache implementation.

>> Drawback: the cache cannot be (easily) controlled across
>> ZEO clients.
>
>One thing why I thought first about using cache objects instead of module  
>namespace was the possibility to share cache between ZEO clients if  
>temporary folder will be changed into persistent folder and mapped with  
>dbtab according to the recipe found at zopelabs.

Yes, that is easier with ZODB based objects.


>Naturally this would make it slower than using module namespace, but it  
>shouldn't it still be faster than doing new queries to catalog.
>
>Any thoughts on that.

It depends.

Many queries are extremely fast. There is no need to cache them.

Sorting large result sets is very expensive (this costs by fast
the most ressources in our application). Caching is valuable
for us for this reason. Therefore, we allow the query to specify whether
it wants to get cached or not (and for how long).

-- 
Dieter