[Zope] Major problems with slow catalog queries

[Casey Duncan]

If you can isolate a case where searching the catalog alone (with no other
operations being performed) takes minutes I would like a chance to analyse
it closer.

Some questions for you first:

How many objects are in your catalog, 11,000?
How many indexes are you searching simultaneously?
What kinds of indexes are these?
Do the searches involve globbing? (* and ? wildcards for text searches).

Globbing searches can be achingly slow using TextIndex/Vocabulary (The
vocabulary is at fault). ZCTextIndex seems to perform much better in this
regard.

-Casey

----- Original Message -----
From: "Lea Smith" <richandleacv@yahoo.com>
To: <zope@zope.org>
Sent: Friday, September 13, 2002 8:01 AM
Subject: [Zope] Major problems with slow catalog queries


> Hi,
>
> I have been developing a site around Zope and the CMF
> and have got to a stage of loading on the data we have
> and testing the site.
> (zope 2.5.1, python 2.1.3, testing on win2000)
>
> The loaded data creates around 11,000 objects which
> are all cataloged in the default portal_catalog. The
> more objects that are added and cataloged the slower
> searches become as well as the whole site in general.
>
> Have used the product callprofiler and found 90% of
> the page creation time is more often than not taken up
> by the script which queries the catalog and returns
> the catalog results. The script does nothing else.
> This time generally varies between 8 to 30 seconds,
> but sometimes as long as 5 minutes!
>
> I can't work out what is going on with the catalog, or
> what is a normal query time for a catalog loaded with
> this amount of info. And the slow response is going to
> get worse as visitors add more content. Not good!
>
> Have tried recreating the catalog just incase
> something had gone wrong there. Played around with
> caching using RAMCache Manager as well as the
> control_panel cache settings with no reliable
> improvement. (Need to read up on the cache usage as
> wasn't quite sure of how best to set it up).
>
> Any ideas on normal response times for the catalog and
> anything I can look at or try would be fantastic.
>
> Thanks for any help.
>
>
> __________________________________________________
> Do You Yahoo!?
> Everything you'll ever need on one web page
> from News and Sport to Email and Music Charts
> http://uk.my.yahoo.com
>
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )
>