[Zope] Cataloging/ignoring objects that generate errors on in dexing

Wed Oct 1 11:32:12 EDT 2003

OK, I've found the code that is causing this to happen:

Zope 2.6.1, in Products/PluginIndexes/TextIndex/TextIndex.py, Line 291:

except (AttributeError, TypeError):

For some reason, the index_object method catches specific exceptions, not
ALL exceptions.  If I change this to a simple "except:", then my problem
goes away.

Conceptually, I would think this is better ... Just ignore ANY error that
might occur and don't index the object ... Why only handle specific
exceptions ?

Is this a bug ?

Thanks,

J.F.

-----Original Message-----
From: Jean-Francois.Doyon at CCRS.NRCan.gc.ca
[mailto:Jean-Francois.Doyon at CCRS.NRCan.gc.ca]
Sent: Monday, September 29, 2003 1:15 PM
To: chrism at zope.com
Cc: zope at zope.org
Subject: RE: [Zope] Cataloging/ignoring objects that generate errors on
in dexing

Hello,

Here is the traceback, but in case that's confusing, find the code that
modifies the CMF Document below as well:

Traceback (innermost last): 

Module ZPublisher.Publish, line 49, in publish 
Module ZPublisher.mapply, line 32, in mapply 
Module ZPublisher.Publish, line 38, in call_object 
Module Products.ZCatalog.ZCatalog, line 292, in manage_catalogFoundItems 
Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply 
Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply 
Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply 
Module Products.CMFCore.CatalogTool, line 226, in catalog_object 
Module Products.ZCatalog.ZCatalog, line 486, in catalog_object 
Module Products.ZCatalog.Catalog, line 299, in catalogObject 
Module Products.PluginIndexes.TextIndex.TextIndex, line 271, in index_object

Module Products.Atlas, line 47, in SearchableText 
Module Products.Atlas, line 34, in CookedBody 
Module Shared.DC.Scripts.Bindings, line 250, in __call__ 
Module Shared.DC.Scripts.Bindings, line 270, in _bindAndExec 
Module Products.PageTemplates.ZopePageTemplate, line 198, in _exec 
Module Products.PageTemplates.PageTemplate, line 79, in pt_render
<ZopePageTemplate at /english/learningresources/quizzes/questions.html/> 
Module TAL.TALInterpreter, line 196, in __call__ 
Module TAL.TALInterpreter, line 229, in interpret 
Module TAL.TALInterpreter, line 633, in do_loop_tal 
Module Products.PageTemplates.TALES, line 201, in setRepeat 
Module Products.PageTemplates.TALES, line 211, in evaluate
URL: 
Line 5, Column 0
Expression: <PythonExpr
here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=int
(request.form['difficulty']),limit=int(request.form['num']))>
Names:
{'container': <Document at
/english/learningresources/quizzes/questions.html>,
 'default': <Products.PageTemplates.TALES.Default instance at 0x8657204>,
 'here': <Document at /english/learningresources/quizzes/questions.html>,
 'loop': <SafeMapping instance at 94b8a60>,
 'modules': <Products.PageTemplates.ZRPythonExpr._SecureModuleImporter
instance at 0x8656fe4>,
 'nothing': None,
 'options': {'args': ()},
 'repeat': <SafeMapping instance at 94b8a60>,
 'request': <HTTPRequest,
URL=http://atlas.gc.ca/newsite/english/portal_catalog/manage_catalogFoundIte
ms>,
 'root': <Application instance at 8d0a9e8>,
 'template': <ZopePageTemplate at
/english/learningresources/quizzes/questions.html/>,
 'traverse_subpath': [],
 'user': admin}
Module Products.PageTemplates.ZRPythonExpr, line 43, in __call__
__traceback_info__:
here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=int
(request.form['difficulty']),limit=int(request.form['num'])) 
Module Python expression
"here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=in
t(request.form['difficulty']),limit=int(request.form['num']))", line 2, in f

Module AccessControl.ZopeGuards, line 75, in guarded_getitem 
KeyError: category

And, in my product's __init__ , I have :

# Monkey Patch to turn CMF Documents into ZPT's, give them Properties, and
fix up the indexing to strip HTML

originalCookedBody = Document.CookedBody
originalManageOptions = Document.manage_options

def CookedBody(self, stx_level=None, setlevel=0):
        # First, run through the original logic
        cookedbody = originalCookedBody(self, stx_level, setlevel)
        # Then if document is HTML, run it through the ZPT rendering engine
        if ( self.text_format == 'html' ):
                pt =
ZopePageTemplate('',cookedbody,'text/html').__of__(self)
                cookedbody = pt()
        return cookedbody

Document.CookedBody = CookedBody

Document.manage_options = ( originalManageOptions +
PropertyManager.manage_options )

def SearchableText(self):
        return "%s %s %s %s" %
(self.Title(),self.Description(),stripper(self.CookedBody()),self.Keywords()
)

Document.SearchableText = SearchableText

...

The thought had occured to me that maybe the catalog only ignores certain
types of exceptions, but I mean ... a KeyError is pretty standard, and
precisely the kind of exception I'd expect the catalog to ignore.

Maybe I can trap the erro and raise some other exception ? Or maybe there's
a bug that exception handling isn't done right ?

Thanks in advance for any help!

J.F.

-----Original Message-----
From: Chris McDonough [mailto:chrism at zope.com]
Sent: Friday, September 26, 2003 2:01 PM
To: Jean-Francois.Doyon at CCRS.NRCan.gc.ca
Cc: zope at zope.org
Subject: Re: [Zope] Cataloging/ignoring objects that generate errors on
indexing

Can you supply a traceback?  I've always been under the same impression
as you....

On Fri, 2003-09-26 at 12:15, Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> Hello,
> 
> I'm trying to update my catalog, and much to my surprise, it ain't working
!
> 
> Some objects require form parameters and so on, raise errors when none are
> provided. So now when I ask the catalog (In this case a CMF one, but the
> fact it's CMF shouldn't really matter, right?) to index everything it can
> find, it raises an error when it lands on such objects.
> 
> Now this has never happened ot me before, and I'd always assumed that
> objects that raised errors while being index simply didn't get indexed,
> hence my suprise. I was actually counting on such doucments NOT getting
> indexed BECAUSE they raise errors.
> 
> Admitedly, I'm doing something somewhat out of the ordinary. I have a CMF
> Document that actually renders/interprets TAL (Basically a CMF Document
> ZPT). SearchableText() runs a CookedBody that does the ZPT rendering ...
Of
> course NOT using CookedBody at indexing might be an option, but I'm not
sure
> of the consequences ... Some documents that have simple
> tal:replace/tal:content type instructions in them I DO want to have Cooked
> before indexing ...
> 
> Anybody have any good ways or suggestions on how to get around this ?
> 
> Any help would be much appreciated!
> 
> Thanks,
> 
> Jean-François Doyon
> Internet Service Development and Systems Support
> GeoAccess Division
> Canadian Center for Remote Sensing
> Natural Resources Canada
> http://atlas.gc.ca
> Phone: (613) 992-4902
> Fax: (613) 947-2410
> 
> 
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
-- 
Chris McDonough <chrism at zope.com>
Zope Corporation

_______________________________________________
Zope maillist  -  Zope at zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )