[Zope3-Users] Re: Community opinion about search+filter

[Martijn Faassen]

Hello,

Adam Groszer wrote:
> I'd like to ask your opinion, your experiences about searching and
> filtering in quite large object DBs.
> We need to add search and filter functions to our current app, where
> the user might be able to create quite _sophisticated_ filter criterias.
> (The app is a pure Z3 app, subject is document management)
> 
> Currently we're looking at something based on catalog/indexes.
> As I checked the most comfortable solution would be based on
> hurry.query.
> Some questions arose:
> - Is it necessary/worth adding indexes on all attributes?
> - How does the index perform on modification and retrieval?
> 
> The biggest problem is that this will be our first try, so we're
> missing experiences and are a bit puzzled about the right solution.
> Certain is that moving to RDB is not an option.

I think one of the main limitations of the current catalog (and 
hurry.query) is efficient support for sorting and batching the query 
results. The Zope 3 catalog returns all matching results, which can then 
be sorted and batched. This will stop being scalable for large 
collections. A relational database is able to do this internally, and is 
potentially able to use optimizations there.

It would be very nice if someone could look into expanding hurry.query 
and/or the catalog to support these cases. It would be interesting to 
look at what Dieter Maurer has done with AdvancedQuery in Zope 2 in this 
regard as well.

Regards,

Martijn