[Zope-CMF] Question, re: folders w/ thousands of items

[sean.upton@uniontrib.com]

I am discovering that, when there are a ton of objects in a folder, skin
methods that use standard ObjectManager (or their CMF-wrapped equivalents)
to list items in a folder are quite expensive compared to getting lists of
items using the portal_catalog tool and PathIndexes to _trying_ and arrive
at the same list (quick but inaccurate, see below); using a Catalog index
and results are much more flexible (I can iterate over the first 20 items in
a folder, for example, in a particular order).  

I do have a question, however: using PathIndexes, I seem to get everything
in an entire folder hierarchy (including items in subfolders), not just
items contained with a particular absolute path (which I want).  Adjusting
the level parameter in searchResults() doesn't seem to get me what I want
(I'm not sure this is what it was designed for, so I'm not surprised).

Any thoughts on what I should do?  Using Catalog results really speeds up
the UI over using OM methods, but I would like to be able to use PathIndexes
to get what I am after (a quick, cached replacement for using
objectValues(), that has sorting, and easy limitation of result sets).  This
leaves me with several questions:

- Am I out of luck, or is there a way to do this with a PathIndex?
- Should I modify/monkey-patch PathIndex, should I implement a new index
plugin based in PathIndex?
- Should I use a field index on a skin method wrapping getPhysicalPath()
that returns a string?  I suppose I can do the latter, but is there a better
way?

>From what I can tell, if it is possible, this sort of Catalog usage is a
pretty good hack^H^H^H^H solution to get better performance from a CMF site
with many items in a single portal folder, and such functionality should
(somehow) be available to anybody coding skins (the assumption, of course,
is that you are willing to assume that all content seen through skins is
Cataloged, and if something is uncataloged, you only see it in the ZMI -
something I'm sure every skin designer could live with).

Thoughts?

Sean

-----Original Message-----
From: sean.upton@uniontrib.com [mailto:sean.upton@uniontrib.com]
Sent: Tuesday, August 06, 2002 5:39 PM
To: zope-cmf@zope.org
Subject: [Zope-CMF] Question, re: folders w/ thousands of items


I have a CMF-based project that I am setting up that happens to have 1000s
of objects within a given folder.  One issue that I have is that most of the
methods called by some skins (I'm using Plon for this project) are way too
expensive... I want to be able to call something like
Folder.contentValues(), but only for a limited subset of items (like, say,
return the first 20 items in the folder)...  Perhaps some alternative to the
standard OM API somewhere to access stuff within either a Portal Folder or
BTreePortalFolder?

Does such a fix exist?

Sean

+-----------------------------------------------------------
| Sean Upton
|  Site Technology Supervisor             SignOnSanDiego.com
|  Development & Integration     The San Diego Union-Tribune
|  619.718.5241                     sean.upton@uniontrib.com
|        PATH_TO_THE_DARK_SIDE = 'c:\winnt\system32'
+-----------------------------------------------------------

_______________________________________________
Zope-CMF maillist  -  Zope-CMF@zope.org
http://lists.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests