[Zope-dev] Catalog performance

Romain Slootmaekers romain at zzict.com
Wed Sep 10 11:50:59 EDT 2003


Nguyen Quan Son wrote:
> Hi,
> I have a problem with performance and memory consumption when trying to do some statistics, using following code:
> ...
> docs = container.portal_catalog(meta_type='Document', ...)
> for doc in docs:
>     obj = doc.getObject()
>     value = obj.attr
>     ...
> 
> With about 10.000 documents this Python script takes 10 minutes and more than 500MB of memory, after that I had to restart Zope. I
> am running Zope 2.6.1 + Plone 1.0 on Windows 2000, Xeon P4 with 1GB RAM.
> What's wrong with this code? Any suggestion is appreciated.
> Nguyen Quan Son.
> 

it's not the catalog that's slow:

t1=time.time()
docs = container.portal_catalog(meta_type='Document', ...)
t2=time.time()
for doc in docs:
      obj = doc.getObject()
      value = obj.attr
      ...
t3=time.time()

print out the times and you'll see that the the finding is fast.
the problem is that you are inflating each and every document one after 
another, and that takes time.

Romain Slootmaekers.


> 
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )
> 
> 





More information about the Zope-Dev mailing list