=?iso-8859-1?Q?Lea_Smith?= writes:
The catalog search times mentioned below are the time the query is with the catalog called from a python script like...
context.do_nothing_script() results = context.portal_catalog( search_dict ) context.do_nothing_script()
The 'do_nothing_script' does just that, but it shows me exactly the time the catalog takes using the CallProfiler Product. How it does this? Looks quite strange for me.
... Passing the following to the portal_catalog causes the catalog to take around 35-45 seconds on average (These queries return 400 results from the 10,500 items cataloged)...
{'modified': {'range': 'min', 'query': DateTime('1995/01/01')}, 'Type': ['Classified'], 'sort_on': 'modified', 'review_state': 'published', 'sort_order': 'reverse'}
Taking out the 'modified' key and passing the following reduces the catalog search time to approx 0.5 to 1 second on average...
{'Type': ['Classified'], 'sort_on': 'modified', 'review_state': 'published', 'sort_order': 'reverse'} I expect, you do not have documents modified before 1995/01/01?
This would mean, the "modified" subquery will return all documents. However, it will construct the resulting set piecemeal, with each value for "modified" adding a single or a few documents only (the chance that they are modified at the same second it not too large). You must expect a quadratic runtime behaviour (in the number of documents) in this case.
... As I wrote this I thought whether cataloging the date as an interger would be any different. Searching the modified date as an interger has reduced the search time from average of 35-45 seconds down to 1.5 - 5 seconds. Still not as fast as I would like, but a whole lot better. Surprising!
Did I not already suggest, you use Zope's profile support ("Contol_Panel --> Debug information --> Profile Information"). Do it, you will clearly see what causes the large time... Dieter