AW: [ZODB-Dev] diploma thesis: ZODB Indexing

[Christian Theune]

Hey,

I'm afraid I can't forget what you mentioned. ;)

Honestly: I'm sorry if I sounded like repelling your ideas. That wasn't
my intention.

I've got the feeling that the current mode of communication is annoying
for you. Unfortunately I'm struggling a bit on coordinating a discussion
that flows both on the list and in our office, I'll try to find a better
mode. Hints and tips are welcome.

Am Mittwoch, den 05.09.2007, 09:55 -0400 schrieb Jim Fulton:
> On Sep 5, 2007, at 9:39 AM, Christian Theune wrote:
>
> > Am Mittwoch, den 05.09.2007, 09:24 -0400 schrieb Jim Fulton:
> >> I'd like to see a generic framework for defining collections and
> >> indexes in Python and querying them efficiently.  No ZODB expertise
> >> should be needed,
> >
> > I have the feeling you already pondered this a bit and have some more
> > specific ideas ... :)
> 
> No more than I've said before.  There is no reason why this has to be  
> zodb specific.

Sebastian and I talked about this face to face and we think we
understood your hint that this doesn't have to be ZODB specific. 

We imagine we need two kinds of components to make this work:

1. A query processor that could look like:

class IQueryProcessor(Interface):

    def query(...):
	"""Returns a list of matching objects. The parameters are
           specific to the query processor in use.
        """

Alternatively, as the signature of the only method isn't specified
anyway, we could make each query processor define its own interface
instead.

2. An object collection that serves two purposes:

a) maintain indexes

b) provide a low-level query API that is rich enough to let different
query processors e.g. for SQL, xpath, ... work against them.

This is the one that needs most work to get the separation of concerns
right. One split we came up with are the responsibilities to define:

- which objects to index
- how to store the indexes
- how to derive the structural relations between objects

Those could be separated into individual components and make the object
collection a component that joins those together.

On the definition of indexes: we're not sure whether a generic set of
indexes will be sufficient (e.g. the three indexes from XISS - class
index, attribute index, structural index) or do those need to be
exchanged? 

For our ad-hoc querying we certainly don't want to have to set up
specialised indexes to make things work, but maybe optional indexes
could be used when possible -- just like RDBMS.


Christian