[Zope-dev] Catalog performance

[Max M]

Nguyen Quan Son wrote:
 > Hi,
 > I have a problem with performance and memory consumption when trying 
to do some statistics, using following code:
 > ...
 > docs = container.portal_catalog(meta_type='Document', ...)
 > for doc in docs:
 >     obj = doc.getObject()
 >     value = obj.attr
 >     ...
 >
 > With about 10.000 documents this Python script takes 10 minutes and 
more than 500MB of memory, after that I had to restart Zope. I
 > am running Zope 2.6.1 + Plone 1.0 on Windows 2000, Xeon P4 with 1GB RAM.
 > What's wrong with this code? Any suggestion is appreciated.
 > Nguyen Quan Son.

Most likely you are filling the memory of your server so that you are 
swapping to disk.

Try cutting the query into smaller pieces so that the memory doesn't get 
filled up.


regards Max M