[Zope-dev] KeywordIndex performance / multiunion

[Casey Duncan]

On Thu, 06 Nov 2003 19:11:55 +0000
Seb Bacon <seb at jamkit.com> wrote:

> A simple query for ["A" or "B" or "C"] against a KeywordIndex containing 
> 27k objects is taking about 7 seconds on a Celeron 1.6Ghz, which seems 
> an absurdly long time to me.

<guess>
This time may be caused by fetching from the database. If so, then the only way to speed it up is increase the ZODB cache or get faster disks. Try the former and see if it helps.
</guess>
 
> The bit of the operation that is taking the time is the part of the 
> index which does a union of the results of each of the 3 query 
> parameters.  Reproducing this BTree union operation outside Zope in a 
> unit test takes milliseconds.  Very occasionally it takes milliseconds 
> inside the KeywordIndex too.  Something to do with memory usage perhaps? 
>   I'll continue looking into it.

Are you saying that sometimes it's faster? If so, that probably supports my guess above. In those times it's all or mostly in memory.
 
> But main the reason I'm posting is to wonder if there any reason not to 
> use the multiunion operator instead of the union operator in 
> UnIndex.py... it should be faster, right?  It seems a touch faster in 
> some informal tests.

Yes, it probably should be used. I think it is much faster when unioning very small and very large sets as well. Tim would know better, tho.

-Casey