John Barratt wrote:
If you can't use catalog metadata as Seb suggests (eg. you are actually accessing many attributes, large values, etc.) and if indeeed memory is the problem (which seems likely) then you can ghostify the objects that were ghosts to begin with, and it will save memory (unless all those objects are already in cache).
The problem with this strategy though is that doc.getObject() method used in your code activates the object and hence you won't know if it was a ghost already or not. To get around this you can shortcut this method and do something like :
docs = container.portal_catalog(meta_type='Document', ...) for doc in docs: obj = doc.aq_parent.unrestrictedTraverse(doc.getPath()) was_ghost = obj._p_changed is None value = obj.attr if was_ghost:obj._p_deactivate()
Just my 2 cents observation... I ran this code and monitored the page "Cache extreme detail" in ZMI > ControlPanel > DebugInfo. With this method, the object was not loaded. However the intermediate objects that the unrestrictedTraverse() passed by were loaded into memory. e.g. If doc.getPath() is '/x/y/z/myobject', myobject was not loaded but x, y, and z were loaded into memory. I also tried the method suggested by Seb. This did not load myobject as well as x, y, and z into memory: http://mail.zope.org/pipermail/zope-dev/2003-September/020450.html The information on deactivating object into ghost state is very helpful. Thanks! cheers, zhi min