[Zope] Re: zcatalog corruption?

[Chris McDonough]

Who knows why this happened; maybe a stray hasattr caught a read
conflict error and a transaction was later committed that had improper
data as a result.  Or maybe it's more mundane, but I doubt unless it
happens continually that you will be able to pin down the cause.

In any case, fixing the symptom surgically isn't simple; it would take
(for me at least) several hours of spelunking the source code and
figuring out which data structures contain what and how they relate to
each other, although it is possible.

The "sledgehammer vs. fly" method is a complete recatalog; that's simple
to do.  You might want to try this, but first back up your Data.fs so in
case it doesn't work or leaves it in an insane state that you can go
back.

If it's not possible to use the sledgehammer method for whatever reason,
it is possible to surgically fix this, it would just take some effort.

HTH,

- C




On Mon, 2004-08-02 at 16:42, nwingfield at dixon-hughes.com wrote:
> 
> 
> I have a large ZCatalog (330,000+ objects) that has been in production for
> nearly a year.  One particularly important index keeps track of client
> names so that users can quickly search by client.  The index is a
> ZCTextIndex called 'clientString'.  Sometime very recently, my query began
> raising an exception when users try to search on the term 'Smith'.  My best
> guess is that one of the records in the catalog containing the word 'Smith'
> has been corrupted somehow.  The query looks like this:
> 
> container.catalogOfEFRObjects.searchResults({'clientString':'Smith'})
> 
> The traceback looks like this:
> 
>    Traceback (innermost last):
>      ...
>      Module Products.ZCatalog.ZCatalog, line 619, in searchResults
>      Module Products.ZCatalog.Catalog, line 751, in searchResults
>      Module Products.ZCatalog.Catalog, line 492, in search
>      Module Products.ZCTextIndex.ZCTextIndex, line 184, in _apply_index
>      Module Products.ZCTextIndex.ParseTree, line 118, in executeQuery
>      Module Products.ZCTextIndex.BaseIndex, line 190, in search
>      Module Products.ZCTextIndex.OkapiIndex, line 160, in _search_wids
>    KeyError: 458693239
> 
> If I look up this key (458693239) in the catalog, I can find a path to the
> object.  If I try to unindex/reindex the object, I get a similar error:
> 
>    Traceback (innermost last):
>      Module ZPublisher.Publish, line 98, in publish
>      Module ZPublisher.mapply, line 88, in mapply
>      Module ZPublisher.Publish, line 39, in call_object
>      Module Products.ElectronicFileRoom.CatalogPathAwareness, line 106, in
>    reindex_object
>      Module Products.ElectronicFileRoom.CatalogPathAwareness, line 102, in
>    unindex_object
>      Module Products.ZCatalog.ZCatalog, line 530, in uncatalog_object
>      Module Products.ZCatalog.Catalog, line 407, in uncatalogObject
>      Module Products.ZCTextIndex.ZCTextIndex, line 166, in unindex_object
>      Module Products.ZCTextIndex.OkapiIndex, line 67, in unindex_doc
>    KeyError: 458693239
> 
> Can anyone offer a diagnosis and/or solution?
> 
> Nathaniel
> 
> ID:[{20040802164223.30403.8035058476-12.6.18.86}]
>