[Zope-Coders] ZCTextIndex too sensitive
Sidnei da Silva
sidnei@x3ng.com
Thu, 12 Jun 2003 15:57:14 -0300
--oLBj+sq0vYjzfsbl
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Thu, Jun 12, 2003 at 01:59:07PM -0400, Casey Duncan wrote:
| > So, how come 'text' is callable but doesnt have __call__? Weird.
|
| It must be trying to index an ExtensionClass instance.
| callable(ExtensionClass instance) always returns true 8^(
Thats what it looks like to me.
| > Now my question: How do we fix this?
| >
| > a. Putting a try/except
| > b. Using 'safe_callable'
| > c. ???
|
| I might suggest using safe_callable, except it would add a dependency to
| DocumentTemplate, which feels very wrong to me.
Just on the start of ZCatalog/Catalog.py there is a try/except that defines a
pythonic version of safe_callable if DocumentTemplate can't be fine,
so I guess that ZCTextIndex depending on ZCatalog is fine? If yes, Ill
just use that.
| The try/except is also wrong
| because you don't know if the exception was the result of calling the thing
| (in which case it should propagate) or a result of it not being callable
| (which usually raises TypeError AFAIK).
In this case it raised an AttributeError. As SteveA explained to me on
irc, it seems like the C extension fills tp_callable slot, but the
class doesnt have a __call__ method. About the try/except being wrong,
I think it needs to be fixed on TextIndex (if someone still uses that
:P) and maybe on other indexes (i havent checked).
| The right answer might be to add an abstracted function that can be imported
| from ZCatalog or PlugInIndexes.common. Something like:
|
| def dataToIndex(obj, name):
| ...get attribute name from obj...
| ...find out if it should be called and call it...
| return data
Not bad. In fact this should be part of the pluggable index interface
(if something like that exists).
| Unfortunately it will be tricky to do this in a way that is compatible with
| Python2.1 and 2.2 (a requirement), works with extension classes (ditto), does
| not mask exceptions and generally does not make everyone queasy (optional
| ;^). On the bright side, all indexes could use this one function making thing
| potentially more consistent.
|
| > PS: May I merge my other patch to KeywordIndexes with the
| > modifications suggested by Casey[1]?
|
| ok by me ;^) Just so long as the tests pass.
Humm.... Running the tests with python2.2.3 I've got one failure:
======================================================================
FAIL: check_unicode_mixed_unknown_encoding
(test_talinterpreter.InterpolateTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_talinterpreter.py", line 148, in
check_unicode_mixed_unknown_encoding
File "/usr/lib/python2.2/unittest.py", line 286, in failUnlessEqual
raise self.failureException, \
AssertionError: u'foo d\xe9j\xe0' != u'foo d\\xe9j\\xe0'
----------------------------------------------------------------------
Ran 1756 tests in 171.015s
FAILED (failures=1)
======================================================================
OVERALL FAILED (total failures=1)
All tests pass with python2.1.3+
Attached is my patch. Ill merge it if there are no objections.
[]'s
--
Sidnei da Silva (dreamcatcher) <sidnei@x3ng.com.br>
X3ng Web Technology <http://www.x3ng.com.br>
GNU/Linux user 257852
Debian GNU/Linux 3.0 (Sid) 2.4.20-powerpc ppc
Overflow on /dev/null, please empty the bit bucket.
--oLBj+sq0vYjzfsbl
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="safe_callable.diff"
Index: lib/python/Products/PluginIndexes/DateIndex/DateIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/DateIndex/DateIndex.py,v
retrieving revision 1.7.6.1
diff -r1.7.6.1 DateIndex.py
16a17
> from types import StringType, FloatType, IntType
21c22
< from types import StringType, FloatType, IntType
---
> from Products.PluginIndexes.common import safe_callable
65c66
< if callable( date_attr ):
---
> if safe_callable( date_attr ):
Index: lib/python/Products/PluginIndexes/DateRangeIndex/DateRangeIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/DateRangeIndex/DateRangeIndex.py,v
retrieving revision 1.4
diff -r1.4 DateRangeIndex.py
16a17,18
> import os
>
19a22
> from Products.PluginIndexes.common import safe_callable
28d30
< import os
159c161
< if callable( since ):
---
> if safe_callable( since ):
164c166
< if callable( until ):
---
> if safe_callable( until ):
Index: lib/python/Products/PluginIndexes/KeywordIndex/KeywordIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/KeywordIndex/KeywordIndex.py,v
retrieving revision 1.11.6.1
diff -r1.11.6.1 KeywordIndex.py
13a14,15
> from types import StringType, UnicodeType
>
20a23
> from Products.PluginIndexes.common import safe_callable
89c92,93
< if callable(newKeywords):
---
>
> if safe_callable(newKeywords):
91,92c95,98
< if hasattr(newKeywords,'capitalize'): # is it string-like ?
< newKeywords = (newKeywords, )
---
>
> if (isinstance(newKeywords, StringType)
> or isinstance(newKeywords, UnicodeType)): #Python 2.1 compat isinstance
> return (newKeywords,)
94d99
< # Uniqueify keywords
96,99c101,108
< for k in newKeywords:
< unique[k] = None
< newKeywords = unique.keys()
< return newKeywords
---
> try:
> for k in newKeywords:
> unique[k] = None
> except TypeError:
> # Not a sequence
> return (newKeywords,)
> else:
> return unique.keys()
Index: lib/python/Products/PluginIndexes/PathIndex/PathIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/PathIndex/PathIndex.py,v
retrieving revision 1.25.6.6
diff -r1.25.6.6 PathIndex.py
17a18
> from Products.PluginIndexes.common import safe_callable
109c110
< if callable(f): path = f()
---
> if safe_callable(f): path = f()
Index: lib/python/Products/PluginIndexes/TextIndex/TextIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/TextIndex/TextIndex.py,v
retrieving revision 1.34
diff -r1.34 TextIndex.py
33a34
> from Products.PluginIndexes.common import safe_callable
285c286
< if callable(source):
---
> if safe_callable(source):
298c299
< if callable(encoding ):
---
> if safe_callable(encoding ):
Index: lib/python/Products/PluginIndexes/common/UnIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/common/UnIndex.py,v
retrieving revision 1.14
diff -r1.14 UnIndex.py
17a18,21
> import sys
> from cgi import escape
> from types import StringType, ListType, IntType, TupleType
>
23d26
< from types import StringType, ListType, IntType, TupleType
32,33c35,36
< import sys
< from cgi import escape
---
> from Products.PluginIndexes.common import safe_callable
>
253c256
< if callable(datum):
---
> if safe_callable(datum):
Index: lib/python/Products/PluginIndexes/common/__init__.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/PluginIndexes/common/__init__.py,v
retrieving revision 1.3
diff -r1.3 __init__.py
1c1,28
< # empty comment for winzip and friends
---
> ##############################################################################
> #
> # Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved.
> #
> # This software is subject to the provisions of the Zope Public License,
> # Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
> # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
> # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
> # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
> # FOR A PARTICULAR PURPOSE
> #
> #############################################################################
>
> # This code is duplicated here from Products/ZCatalog/Catalog.py to avoid a
> # unnecessary dependency on ZCatalog.
>
> try:
> from DocumentTemplate.cDocumentTemplate import safe_callable
> except ImportError:
> def safe_callable(ob):
> # Works with ExtensionClasses and Acquisition.
> if hasattr(ob, '__class__'):
> if hasattr(ob, '__call__'):
> return 1
> else:
> return isinstance(ob, types.ClassType)
> else:
> return callable(ob)
Index: lib/python/Products/ZCTextIndex/ZCTextIndex.py
===================================================================
RCS file: /cvs-repository/Zope/lib/python/Products/ZCTextIndex/ZCTextIndex.py,v
retrieving revision 1.35.2.7
diff -r1.35.2.7 ZCTextIndex.py
32a33
> from Products.PluginIndexes.common import safe_callable
156c157
< if callable(text):
---
> if safe_callable(text):
--oLBj+sq0vYjzfsbl--