[Zope3-Users] Search multiple fields with TextIndex

Tue Feb 17 17:50:35 EST 2009

Massimiliano della Rovere wrote:
> Another question is:
> is there a way to automatically create TextIndices for certain (all) fields
> of an interface?

No, I think there's not. The event subscriber to IEventoNuovoSiteMabon
is the way to go.
Instead of using a field manager :) I suggest you use the api of
zope.interface and zope.schema to iterate over the fields and drop some.
See p.64 of Philipp's book (third edition).

form zope.app.catalog.text import TextIndex

def catalogo_e_indice(event):
  gs = event.object.getSiteManager()

  ... # create intid utility and catalog as you did

  for campo in zope.schema.getFieldNames(IScheda):
    if not campo in ('__name__', '__parent__', 'some_field_I_want_to_drop'):
      catalogo[campo] = TextIndex(
	interface = IScheda,
	field_name = campo,
	field_callable = False)

> 
> On Mon, Feb 16, 2009 at 17:23, Massimiliano della Rovere <
> massimiliano.dellarovere at gmail.com> wrote:
> 
>> Probably because I am a zope newbie, I have some problems in understanding
>> how queries are managed.
>>
>> I am reading von Weithershausen book, and I notice that the
>> SearchPage.__call__ method takes the query parameter. How does zope know
>> that the query (I guess it is request.query or request.form.query; I wish
>> zope had an interactive debugger with inspector) paramter fromthe search
>> viewlet must be passed to the __call__method?

I wonder about that, too.

>> parameter name matching?

It is a browserpage, see zope.publisher.interfaces.browser for IBrowserPage.

>>
>> I have the usual interface with the autohor, subject and location fields
>> (the real class has over 30 fields).
>> I created a search page with a form containing a field for each attribute,
>> implemented a class derived from FormBase and action attribut of the form
>> tag pointing to the view that should perform the search.
>>
>> I am thinking how the request to the catalogo.searchResults should receive
>> the parameter ttyped by the user (there could be less parameter than the
>> search fields).
>>

If you want to use the form framework have have to decide to use a
method that get's called after the input was validated as a hook to call
query the catalog and a new template to show the result. You could use
the add-method (or createAndAdd-method) as this hook.

But I think the form-framework (formlib or z3c.form) are a not felxible
enough for complex search forms. It would be nice if it was possible to
query one index, invert queries, choose a conjunction, and the like. It
is easy to write form interaction in zope. The input is on the request
attribute.

I attached viewclasses and templates from my app. Well--my catalog has
has less indices than yours :) .. Feel free reuse it.
There are two tar balls. One (search-form.tgz) with more abstract
classes that can be used to derive search forms for a catalog. It
validates the input against the schema-fields from the interface and
raises a UserError on failure. It works for textindices only, now. Value
indices need some more logic.. I have some plans about additional
features, but that's not implemented yet..
The other (example-search.tgz) uses these classes for searching for
examples...

>> I implemented a few bits to try to realize what I wrote.
>> We have the class preparing the catalog and indices when a site istance is
>> created:
>> @adapter(IEventoNuovoSitoMabon)
>> def catalogo_e_indice( evento ):
>> gs = evento.object.getSiteManager()
>>
>> intids = IntIds()
>> gs['intids'] = intids
>> gs.registerUtility( intids, IIntIds )
>>
>> catalogo = Catalog()
>> gs['catalogo'] = catalogo
>> gs.registerUtility( catalogo, ICatalog )
>>

That looks OK.

>> campi = [ i.__name__ for i in Fields(IScheda) ] #.omit('__name__',
>> '__parent__', 'titolo') ]
>>

Do you use a field manager?

>> for i in campi:
>> testuale = TextIndex(
>> interface = ISearchableText,
>> field_name = i,
>> field_callable = False
>> )
>> catalogo[i] = testuale
>>
>> semplice = TextIndex(
>> interface = ISearchableText,
>> field_name = 'getSearchableText',
>> field_callable = True
>> )
>> catalogo[u'semplice'] = semplice
>>
>>
>> and the data extractor:
>> class TestoSchedaPerRicerca(object):
>> """Estrattore del testo per le ricerche"""
>> implements(ISearchableText)
>> adapts(IScheda)
>>
>> def __init__( self, context ):
>> self.context = context
>>
>> def __getattr__( self, attr ):

# What is this method for? Does ISearchableText subclass IScheda?

>> try:
>> return self.context.__dict__[attr]
>> except KeyError:
>> raise AttributeError( attr )
>>
>> def getSearchableText( self ):
>> return self.context.autore + u' : ' + self.context.soggetto
>>
>> But now when I add a Scheda object I get the following error:
>>
>> 2009-02-16T17:17:33 ERROR SiteError
>> http://localhost:9673/mabon/schede/+/mabon.Scheda%3D
>>
>> Traceback (most recent call last):
>>
>>  File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line
>> 133, in publish
>>    result = publication.callObject(request, obj)
>>
>>  File
>> "/usr/lib/python2.4/site-packages/zope/app/publication/zopepublication.py",
>> line 161, in callObject
>>    return mapply(ob, request.getPositionalArguments(), request)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line
>> 108, in mapply
>>    return debug_call(obj, args)
>>
>>   - __traceback_info__: <security proxied
>> zope.app.publisher.browser.viewmeta.ModuloAggiungiScheda instance at
>> 0xa87036c>
>>  File "/usr/lib/python2.4/site-packages/zope/publisher/publish.py", line
>> 114, in debug_call
>>    return obj(*args)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 769, in
>> __call__
>>    self.update()
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 750, in
>> update
>>    result = action.success(data)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 594, in
>> success
>>    return self.success_handler(self.form, self, data)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 861, in
>> handle_add
>>    self.createAndAdd(data)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 868, in
>> createAndAdd
>>    return self.add(ob)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/formlib/form.py", line 877, in
>> add
>>    ob = self.context.add(object)
>>
>>  File
>> "/usr/lib/python2.4/site-packages/zope/app/container/browser/adding.py",
>> line 72, in add
>>    container[name] = content
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/container/sample.py", line
>> 86, in __setitem__
>>    setitem(self, self.__data.__setitem__, key, object)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/container/contained.py",
>> line 593, in setitem
>>    notify(event)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/event/__init__.py", line 23,
>> in notify
>>    subscriber(event)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/event.py", line 26,
>> in dispatch
>>    for ignored in zope.component.subscribers(event, None):
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/_api.py", line 130,
>> in subscribers
>>    return sitemanager.subscribers(objects, interface)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/registry.py", line
>> 290, in subscribers
>>    return self.adapters.subscribers(objects, provided)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/interface/adapter.py", line
>> 535, in subscribers
>>    subscription(*objects)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/event.py", line 33,
>> in objectEventNotify
>>    adapters = zope.component.subscribers((event.object, event), None)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/_api.py", line 130,
>> in subscribers
>>    return sitemanager.subscribers(objects, interface)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/registry.py", line
>> 290, in subscribers
>>    return self.adapters.subscribers(objects, provided)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/interface/adapter.py", line
>> 535, in subscribers
>>    subscription(*objects)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/intid/__init__.py", line
>> 169, in addIntIdSubscriber
>>    notify(IntIdAddedEvent(ob, event))
>>
>>  File "/usr/lib/python2.4/site-packages/zope/event/__init__.py", line 23,
>> in notify
>>    subscriber(event)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/event.py", line 26,
>> in dispatch
>>    for ignored in zope.component.subscribers(event, None):
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/_api.py", line 130,
>> in subscribers
>>    return sitemanager.subscribers(objects, interface)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/component/registry.py", line
>> 290, in subscribers
>>    return self.adapters.subscribers(objects, provided)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/interface/adapter.py", line
>> 535, in subscribers
>>    subscription(*objects)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/catalog/catalog.py", line
>> 153, in indexDocSubscriber
>>    cat.index_doc(id, ob)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/catalog/catalog.py", line
>> 62, in index_doc
>>    index.index_doc(docid, texts)
>>
>>  File "/usr/lib/python2.4/site-packages/zope/app/catalog/attribute.py",
>> line 144, in index_doc
>>    return super(AttributeIndex, self).index_doc(docid, value)
>>  File "/usr/lib/python2.4/site-packages/zope/index/text/textindex.py", line
>> 45, in index_doc
>>    self.index.index_doc(docid, text)
>>  File "/usr/lib/python2.4/site-packages/zope/index/text/okapiindex.py",
>> line 225, in index_doc
>>    count = BaseIndex.index_doc(self, docid, text)
>>  File "/usr/lib/python2.4/site-packages/zope/index/text/baseindex.py", line
>> 95, in index_doc
>>    wids = self._lexicon.sourceToWordIds(text)
>>  File "/usr/lib/python2.4/site-packages/zope/index/text/lexicon.py", line
>> 66, in sourceToWordIds
>>    for t in last:
>> TypeError: iteration over non-sequence
>>

Hmm, I don't know. Which class did you instanciate your indices form?
Try zope.app.catalog.text.TextIndex

Hope, that helps.

Regards,
Christian

PS. Please cc your mail to the list!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: example-search.tgz
Type: application/x-compressed-tar
Size: 2218 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope3-users/attachments/20090217/fbe1dd35/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: search-form.tgz
Type: application/x-compressed-tar
Size: 7480 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope3-users/attachments/20090217/fbe1dd35/attachment-0001.bin 