[Zope-dev] Re: Caching ZCatalog results

[Dieter Maurer]

Roché Compaan wrote at 2007-2-23 22:00 +0200:
> ...
>Thanks for that pointer. It's good that way, it should make invalidation
>easier. It could be as simple as invalidating any cached result that
>contains the documentId being indexed. Do you see any problem with the
>following invalidation strategy:
>
>If the 'documentId' exists (cataloging existing object), invalidate all
>cached result sets that contain the documentId.
>
>If the 'documentId' doesn't exist (cataloging new object), invalidate
>all result sets where the ids of indexes applied, are contained in the
>cache key for that result set.

I see several problems:

  *  the RAMCacheManager does not provide an API to implement
     this policy

  *  a cache manager would need a special data structure
     to efficiently implement the policy (given a documentId,
     find all cached results containing the documentId).

  *  Apparently, your cache key contains the indexes involved
     in producing the result.

     The problem with this is that these indexes are known
     only after the query has been performed:

        The catalog API allows indexes to respond to subqueries,
        that do not contain their own name.

        I use this feature to allow a "Managable RangeIndex"
	to transparently replace "effective, expires" queries.

	But otherwise, the feature is probably not used
	intensively.

     Of course, you can add the information *after*
     the query has been performed and use it for invalidation -- in
     a specialized cache manager.


     On the other hand, new objects are usually indexed with
     all available (and not only a few) indexes.

     While some of the indexes may not be able to determine
     a senseful value for the document, the standard indexes
     have problems to handle this properly ("ManagableIndex"es can)
     and the API does not propagate the information.



-- 
Dieter