[Zope-dev] catalog performance: query plan

[Lennart Regebro]

On Mon, Oct 27, 2008 at 17:32, Alan Runyan <runyaga at runyaga.com> wrote:
> I agree with Tres.  A lot more can be done with Indexes and Catalog
> without caching.
>
> The most exiciting development in Catalog optimizations comes out
> Jarn.  Helge Tesdal (iirc) did a buncha work  at a RDBMS company when
> he was in college.  He has a protoype of a query plan for ZCatalog.
>
> http://www.jarn.com/blog/catalog-query-plan
>
> I would like to ask Roche and others to look at the Query Plan.
>
> Caching is a total PITA because invalidation machinery becomes
> overwhelming complex and unwieldly quickly in production.

I don't know very much about searching, but this definitely sounds
like a good idea. Also, especially when doing free text searching that
has large result sets, incremental searching is very beneficial. I
know Dieter Maurer has made a Zope2 implementation of this.

I would be interested in seeing a bunch of Gurus sit down at some
sprint and trying to come up with a catalog engine that is incremental
and uses query plans. There is no reason that would not be stupidly
fast. :) We can then make a new catalog that uses this engine but has
the same API as the old one, to ship with some future version of Zope,
say 2.12.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64