[Zope3-Users] Re: zc.tables integration with Zope3

[Luis De la Parra]

Hello

this is how indexes are implemented in RDBMs.. don't know if sqllite has
such a feature, but in oracle the table content is just stored in files
(structured, but unordered) and a set of sorted indexes is kept for the
primary key and all other secundary indexes. 

each index has the search fields like index0 = "lastname/firstname" and
index1 = "telephone" and a  file/segment/row pointer to the information.
every update to the table triggers an update to all indices in the same
transaction, but you get searching, sorting and batching in return.

cheers. luis


>> 
>> The idea is that you can have indexes to several permutations
>> (first-name-sort, last-name sort, last-modified-date-sort, etc) of a
>> set, and refer to those permutations by name, which refers to an integer
>> index.  The factoradic representation of that integer can be used in a
>> quick algorithm to make a list of the ids in the set in that specific
>> order, and simply slicing that list would give easy batching capability
>> without needing to access the objects at all.  There would, however, be
>> a need to update all of the permutation indexes when the data change.
>> 
>> One thing I have not dealt with yet to make this really useful would be
>> a utility that memoizes the factorials and factoradics, which are
>> expensive to calculate, probably using an sqlite database.
>> 
>> The big trade-off is doing a sort each time the data is accessed vs.
>> doing all of the named sorts each time the data is changed.
>> 
>> I am not sure at the moment whether this solves a real need yet, but I
>> thought I would bring it up since we are discussing batching.  I think
>> it is just at the "interesting idea" stage for right now.
>> 
>> -Jim Washington
>>