[Zope] Zope performing poorly after adding objects to BTreeFo lder

[sean.upton@uniontrib.com]

Your problem sounds like a memory leak.  I think it has to do with
BTreeFolder or one of its parent classes, and not Catalog.

I think there are ups and downs to BTreeFolder.  BTreeFolder does well for
storing lots of objects, but the ObjectManager interface that all folders
use via mixin doesn't scale very well in certain respects for handling all
those objects (at least that is my experience with
ObjectManager._setObject(), so I instead used BTreeFolder._setOb() which
yielded a HUGE performance increase; I figure the same may be true with
_getObject(), etc).

I would suggest familiarizing yourself with the BTree interfaces and using
BTreeFolder._tree() directly.  This has been my experience creating a
container to store ~330,000 objects in a BTreeFolder.  Of course, this
bypasses security, so careful planning and development is advised.

I don't think ZCatalog is your problem, as it sounds like the Catalog is not
being used in certain cases (like listing objects via objectIds() or
objectValues()), and you still get a problem.

Personally, I wonder if BTreeFolder really needs more work to override more
OM interfaces with 'native' faster versions.  I'm interested in hearing
other's thoughts...

Sean

-----Original Message-----
From: complaw@hal-pc.org [mailto:complaw@hal-pc.org]
Sent: Wednesday, January 02, 2002 3:35 PM
To: zope@zope.org
Subject: [Zope] Zope performing poorly after adding objects to
BTreeFolder


I don't know if this is related specifically to BTreeFolder, or cataloging,
but 
this is the problem...

I added a BTreeFolder to a regular folder.  I then added 13,115 DTML
Documents 
to the BTreeFolder.  I had added a ZCatalog and got it going, but the 
performance hit from the ZCatalog made me think twice about it so I deleted
it.

Unfortunately, some python process (related to Zope) starts running on my 
machine when I access the 13,115 records (if nothing more than to list
them).  
I can watch the python process start eating memory (Zope normally uses about
8% 
on that system, but when the runaway process is going, it moves up 
incrementally to around 80% of memory according to top).  I have to restart 
Zope to cure the problem.  Unfortunately, I have to do this on a regular
basis 
and it is becoming quite a headache.

Does anyone know what the problem might be?

Thanks in advance,

Ron


_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )