Has anyone done any benchmarking of how long the various flavors of text indexes for the catalog (ZCTextIndex vs TextIndex vs TextIndexNG) take to index themselves? I'm still not entirely convinced my memory leak is fixed, and I've definitely identified our single biggest performance problem as when PrincipiaSearchSource is indexed. I'm considering using two catalogs, one of which indexes categories and other things that we use for static queries and a second to handle archiving full text. The first would update with the Catalog-awareness, but the full-text would be indexed on a cron. I'm not sure about that, though--I've got to find a way to say "full-text index any Zclass instances that aren't in the full-text index already." I guess I can schedule catalog "finds" based on a date, right? Thinking out loud again... Chris muldrow@mac.com
I can only speak for TextIndexNG. When you use it with the standard setting it is 50-75% faster than the old TextIndexNG. But performance will drop when you use features like stopwords, normalization, stemming etc. -aj --On Mittwoch, 29. Januar 2003 09:06 -0500 Chris Muldrow <cmuldrow@fredericksburg.com> wrote:
Has anyone done any benchmarking of how long the various flavors of text indexes for the catalog (ZCTextIndex vs TextIndex vs TextIndexNG) take to index themselves? I'm still not entirely convinced my memory leak is fixed, and I've definitely identified our single biggest performance problem as when PrincipiaSearchSource is indexed. I'm considering using two catalogs, one of which indexes categories and other things that we use for static queries and a second to handle archiving full text. The first would update with the Catalog-awareness, but the full-text would be indexed on a cron. I'm not sure about that, though--I've got to find a way to say "full-text index any Zclass instances that aren't in the full-text index already." I guess I can schedule catalog "finds" based on a date, right?
Thinking out loud again... Chris muldrow@mac.com
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
--------------------------------------------------------------------- - Andreas Jung http://www.andreas-jung.com - - EMail: andreas at andreas-jung.com - - "Life is too short to (re)write parsers" - ---------------------------------------------------------------------
Chris, FWIW, you may want to look at the QueueCatalog product at http://cvs.zope.org/Products/QueueCatalog . It is a Catalog implementation which defers indexing until a queue is filled.. That said, I doubt it has docs. - C On Wed, 2003-01-29 at 09:06, Chris Muldrow wrote:
Has anyone done any benchmarking of how long the various flavors of text indexes for the catalog (ZCTextIndex vs TextIndex vs TextIndexNG) take to index themselves? I'm still not entirely convinced my memory leak is fixed, and I've definitely identified our single biggest performance problem as when PrincipiaSearchSource is indexed. I'm considering using two catalogs, one of which indexes categories and other things that we use for static queries and a second to handle archiving full text. The first would update with the Catalog-awareness, but the full-text would be indexed on a cron. I'm not sure about that, though--I've got to find a way to say "full-text index any Zclass instances that aren't in the full-text index already." I guess I can schedule catalog "finds" based on a date, right?
Thinking out loud again... Chris muldrow@mac.com
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) -- Chris McDonough <chrism@zope.com> Zope Corporation
On 1/29/03 9:56 AM, "Chris McDonough" <chrism@zope.com> wrote:
Chris,
FWIW, you may want to look at the QueueCatalog product at http://cvs.zope.org/Products/QueueCatalog . It is a Catalog implementation which defers indexing until a queue is filled.. That said, I doubt it has docs.
This is exactly what I need for my indexing plan!! Very cool.
participants (3)
-
Andreas Jung -
Chris McDonough -
Chris Muldrow