[ZWeb] ZCatalog Issues

Andreas Jung lists at andreas-jung.com
Sat Jul 10 03:09:18 EDT 2004


We are having the very same problems with our new company CMS which uses 
several
ZEO storage with up to 400k objects in *each* partition. In some case it is 
necessary for us
to reindex only a small number of the documents that match a particular 
criteria..I am also
interested to help out and improve the handling of such large catalogs in a 
way that I could
sleep calm during a nightly reindexing session :-)

Andreas

--On Freitag, 9. Juli 2004 15:43 Uhr -0700 Michael Bernstein 
<webmaven at cox.net> wrote:

> Here is a summary of a chat this morning on #zope-web where some
> long-existing ZCatalog issues were aired out. They are listed in
> dependency order:
>
> 1. It takes a *very* long time to view the indexes tab (this was
> something I thought Casey had fixed, but the problem seems to have
> re-emerged). This issue isn't actually blocking anything, it just makes
> it incredibly annoying and time consuming to deal with the Catalog at all.
>
> 2. Attempting to re-index the ZCatalog (by clicking the 'Update Catalog'
> button in the 'Advanced' tab) causes a timeout or an error message. There
> are over 179k objects, but it is my understanding that ZCatalog should
> scale to that many objects fairly easily. I am given to understand that
> Casey and Shane have been working on fixing this issue.
>
> Possible workaround:
> - Reindex the individual objects one at a time or in very small batches
> by connecting to the ZODB with a ZEO client (suggested by Simon Michael
> and Andrew Sawyer).
>
> 3. Due to #2, it is as yet impractical to add new indexes to the ZCatalog
> in order to solve various issues with the site. For example, I added a
> title_or_id FieldIndex to the ZCatalog in early May to help solve a
> problem with the Product listings. As of now, this index is only aware of
> 6211 objects out of the 179k. These are only the objects that have been
> added or edited since I created the index. As a result, I can't solve the
> Products issue (though I have hidden it in the meantime). I have also
> been reluctant to tackle the HowTo listings until the ZCatalog problems
> are solved (or some reasonable facsimile thereof) for the same reason.
>
> --
> - Michael R. Bernstein  |  Author of Zope Bible
>    michaelbernstein.com  |  & Zope.org Webmaster
>       panhedron.com      |    PythonPhotos.org
>
> _______________________________________________
> Zope-web maillist  -  Zope-web at zope.org
> http://mail.zope.org/mailman/listinfo/zope-web






More information about the Zope-web mailing list