[Zope3-Users] zc.tables integration with Zope3

[David Pratt]

Hi Jim. This approach is completely new to me. It is interesting and 
I'll keep an eye on your blog. How expensive is updating indexes with 
data modification. You are talking about sqlite, does this mean you 
figure the indexes would be a significant drag on the ZODB due their 
size or is this idea a performance consideration. Many thanks.

Regards,
David


Jim Washington wrote:
> This is probably a bit premature, but I have been doing some thinking 
> about reordering/batching of large sets.
> 
> My current (not-quite-ready-for-prime-time) solution involves 
> factoradics, and I have done a bit of a write-up on my blog, 
> http://blog.hill-street.net/?p=5 .
> 
> The idea is that you can have indexes to several permutations 
> (first-name-sort, last-name sort, last-modified-date-sort, etc) of a 
> set, and refer to those permutations by name, which refers to an integer 
> index.  The factoradic representation of that integer can be used in a 
> quick algorithm to make a list of the ids in the set in that specific 
> order, and simply slicing that list would give easy batching capability 
> without needing to access the objects at all.  There would, however, be 
> a need to update all of the permutation indexes when the data change.
> 
> One thing I have not dealt with yet to make this really useful would be 
> a utility that memoizes the factorials and factoradics, which are 
> expensive to calculate, probably using an sqlite database.
> 
> The big trade-off is doing a sort each time the data is accessed vs. 
> doing all of the named sorts each time the data is changed.
> 
> I am not sure at the moment whether this solves a real need yet, but I 
> thought I would bring it up since we are discussing batching.  I think 
> it is just at the "interesting idea" stage for right now.
> 
> -Jim Washington
>