On Thu, 5 Apr 2001, Michael R. Bernstein wrote:
I'm trying to find out of there is a point where you start getting non-linear performance penalties for additional objects (storing, retreiving, or indexing).
I don't know, but I feel that is the case. Actually, I know it is the case, but I don't know what is causing it. I know what isn't helping though; CatalogAwareness. I added 2000 objects with XML-RPC. No other queries were done during that period. For each object about 70 DTML Method/Documents were added. The first couple of hundres went with a pace of 2-3 seconds per object. After that it started to get real slow, and when I reached about 500 I was down to 5 seconds per object. I killed that script, rewrote it to only add 20-25 DTML Methods/Documents and removed the CatalogAwareness and whoosh! Under 1 second for each object and it stayed like that for the entire 2000 objects. The server is a 1GHz thingy with 1GB RAM. It wasn't working too hard - it seemed.
Meanwhile Erik, what approach *did* your programmer take?
Well, the obviously more correct one. :) He just made the files (that I were going to index in a Catalog) stay on the filesystem and wrote some nice regexps to do the searching I though I needed the speed of the Catalog to do (yeah, yeah, I'm a rookie). Thanks Jim! :)
I'll look forward to it.
Ok, and you know what to do if you haven't heard from me and the year is not 2001 any more ;)