[Zope] ZCatalog speed?

[Stuart Woolford]

On Mon, 13 Sep 1999, Paul Everitt wrote:
> Stuart Woolford wrote:
> > a - 22 minutes to create 8800 documents (smallish) in 1200 folders within zope,
> > not too fast :( but not exactly a user-interaction-limiting factor :)
> 
> Yes, ZCatalog right now is tuned for incremental indexing and
> retrieval.  There is a knob where you can trade memory during indexing
> for speed of indexing.  This works by decreasing the frequency of
> subtransaction commits (which is where the performance hit lies).

I'll have a look at that, currently I'm trying to redo the stuff I'm going to
index using ZClasses to cut down on the raw size..
> 
> > b - too long to then try to do a search based add to a zcatalog,
> > ie: netscape times out after only around 8 minutes, and the search has not
> > finished!
> 
> This is absolutely, obviously something wrong with your ZCatalog
> settings.  Make sure that you have indexed the properties that you are
> searching on, otherwise it will be a "grep" style search.

I'm not talking about the actual searching using the ZCatalog, but trying to do
an add to the ZCatalog, using the 'search for' system to locate the items to
add to the catalog..
 >  > I have seen ZCatalog go through a 12 Mb, 23 thousand
line RDF version of > the RPM repository.  Search time was less than a tenth of
a second. > 

Once I get the items indexed, I bet it will rock!

> --Paul

--
------------------------------------------------------------
Stuart Woolford, stuartw@newmail.net
Unix Consultant.
Software Developer.
Supra Club of New Zealand.
-----------------------------------------------------------