kevin7kal wrote at 2006-12-6 14:30 -0500:
I have a database that is just under 200K objects with plans that it will grow larger. The objects are created by using another object with methods that read files and create the objects from the data in the files. When creating the objects, I find I have to manually clear the cache or else zope eventually crashes with an out of memory error. There are 4GB of ram on the server. The same issue arises when I catalog the objects. I am trying maintain as much speed as possible, so I do not want to catalog until the end of the import.
Is there a way to programmatically clear the cache during a large indexing to avoid out of memory errors?
A ZODB "Connection" has the method "cacheGC()" which you can call to start a cache garbage collection. As others already pointed out, it is implicitely called during large catalog operations -- controlled by the so called "subtransaction threshold". Note that in early versions of ZODB 3.4, there was a bug preventing the call to "cacheGC" at "subtransaction" boundaries. -- Dieter