[ZODB-Dev] Re: AW: diploma thesis: ZODB Indexing

[Laurence Rowe]

Christian Theune wrote:
<snip />
> We imagine we need two kinds of components to make this work:
> 
> 1. A query processor that could look like:
> 
> class IQueryProcessor(Interface):
> 
>     def query(...):
> 	"""Returns a list of matching objects. The parameters are
>            specific to the query processor in use.
>         """
> 
> Alternatively, as the signature of the only method isn't specified
> anyway, we could make each query processor define its own interface
> instead.
> 
> 2. An object collection that serves two purposes:
> 
> a) maintain indexes
> 
> b) provide a low-level query API that is rich enough to let different
> query processors e.g. for SQL, xpath, ... work against them.
> 
> This is the one that needs most work to get the separation of concerns
> right. One split we came up with are the responsibilities to define:
> 
> - which objects to index
> - how to store the indexes
> - how to derive the structural relations between objects
> 
> Those could be separated into individual components and make the object
> collection a component that joins those together.
> 
> On the definition of indexes: we're not sure whether a generic set of
> indexes will be sufficient (e.g. the three indexes from XISS - class
> index, attribute index, structural index) or do those need to be
> exchanged? 
> 
> For our ad-hoc querying we certainly don't want to have to set up
> specialised indexes to make things work, but maybe optional indexes
> could be used when possible -- just like RDBMS.
>

Make sure you take a look at SQLAlchemy's implementation of this, 
sqlalchemy.orm.query.

RDBMS do not get fast querying for free... They just revert to a 
complete record scan when they do not have an index - analogous to the 
find tab in the ZMI. As anyone who has ever queried such a database can 
attest, it ain't quick. (RDBMSs tend to create implicit indexes on 
primary and foreign keys also.)

Laurence