[Zope-dev] Catalog performance

[zhimin at iss.nus.edu.sg]

John Barratt wrote:

> If you can't use catalog metadata as Seb suggests (eg. you are actually
> accessing many attributes, large values, etc.) and if indeeed memory is
> the problem (which seems likely) then you can ghostify the objects that
> were ghosts to begin with, and it will save memory (unless all those
> objects are already in cache).
>
> The problem with this strategy though is that doc.getObject() method
> used in your code activates the object and hence you won't know if it
> was a ghost already or not.  To get around this you can shortcut this
> method and do something like :
>
> docs = container.portal_catalog(meta_type='Document', ...)
> for doc in docs:
>      obj = doc.aq_parent.unrestrictedTraverse(doc.getPath())
>      was_ghost = obj._p_changed is None
>      value = obj.attr
>      if was_ghost:obj._p_deactivate()

Just my 2 cents observation...
I ran this code and monitored the page "Cache extreme detail" in ZMI >
ControlPanel > DebugInfo.
With this method, the object was not loaded. However the intermediate
objects that the unrestrictedTraverse() passed by were loaded into memory.
e.g. If doc.getPath() is '/x/y/z/myobject', myobject was not loaded but x,
y, and z were loaded into memory.

I also tried the method suggested by Seb. This did not load myobject as
well as x, y, and z into memory:
http://mail.zope.org/pipermail/zope-dev/2003-September/020450.html

The information on deactivating object into ghost state is very helpful.
Thanks!

cheers,
zhi min