[ZODB-Dev] Using Catalog and BTrees

[Christian Robottom Reis]

On Tue, 22 May 2001, Michel Pelletier wrote:

> On Tue, 22 May 2001, Christian Robottom Reis wrote:
>
> > If you (or Jim) can point me out to what needs to be done, I'll be busy
> > reading and hacking away to get something useful.
>
> I honestly couldn't tell you where to begin, perhaps you should try
> getting some goals, requirements, use cases, etc?  Then you'll know more
> about your problem.

StandaloneCatalog

Goal: The facility exists to allow efficient searching through
collections of objects. Currently there is no standard way to create
indexes and perform efficient searches through collections of objects,
though SearchIndex and ZCatalog do provide a reference implementation.

Requirements: We need to permit searching collections of objects by
attributes; depending on the type of attributes we would like to
parameterize the searches using different operators. Examples of operators
would be regex for strings, less than/more than for integer, fp and
complex numbers, and equality for object references.

The current implementation of ZODB, using pickles, promotes a need for
indexing objects by attribute. The facility would thus provide mechanisms
for creating and deleting indexes. These indexes ideally would be
maintained automatically - addition of an object to a collection would
automatically update the indexes to reflect the new object's attributes,
and removal would trigger an analogous de-index action.

The object collections may be stored in data structures that are commonly
used in association with the ZODB: Python hashes and lists, and Zope
BTrees. The combination of data structure + indexes + index management
intelligence would compose the proposed facility.

This facility must work outside of Zope to be of assistance to users of
ZODB that are not Zope users.

My question is: is changing SearchIndex and Catalog to fit into these
requirements (if they don't already) a good idea? Or should I write
something (possibly derived) incompatible?

> No, an index is a persistent object that sits inside the same ZODB as the
> persistent objects it indexes.  If you change one object, ZODB does not
> know, or care, that that object is indexes in some other object so it will
> not automagically do anything.

And what does this have to do with indexes keeping track of the objects
they index? This is external to the ZODB; IMHO it's what the Catalog
should do in theory, though it's not straightforward to use outside of
Zope. That's what I'm arguing for.

Take care,
--
/\/\ Christian Reis, Senior Engineer, Async Open Source, Brazil
~\/~ http://async.com.br/~kiko/ | [+55 16] 274 4311