[Grok-dev] Re: Sorting catalog results

[Martijn Faassen]

Luciano Ramalho wrote:
> Trying to figure out how to sort results from a catalog search, I just
> read (most of) a very long thread on Zope3-dev earlier this year and
> got worried about how it ended.
> 
> Does it mean that I have to educate my users ("You don't really want
> sorted results, on account of scalability problems"), or is there some
> recommended way to sort the results of a catalog search in Zope3 or
> Grok?

No, please don't educate your users. The debate went back and forth. I 
think we all agree that the sorting story could be scaled better, but 
Jim kept pointing out there is no fundamental way to speed it up, and I 
kept pointing out that besides the fundamentals there may be many things 
we can do to make this scale better nonetheless. We got stuck in a loop 
there. :)

There are two strategies here. One is the short-term 
getting-things-done-strategy. For that, I'd recommend using Python's (or 
zc.table's, if you're using that for tabular display) sorting 
functionality. That sorts the whole result set. It may scale well enough 
for your application.

Now on to the other strategy. Ignas has done some work on the SchoolTool 
project concerning scalable sorting and batching that may be relevant 
here and reported to me that he managed to speed things quite a lot. I 
don't know the details, but here are pointers to the code he gave me a 
while ago:

http://source.schooltool.org/trac/browser/trunk/schooltool/src/schooltool/skin/table.py
- FilterWidget and TableFormatter classes

http://source.schooltool.org/trac/browser/trunk/schooltool/src/schooltool/skin/table.py
- TableContainerView class

http://source.schooltool.org/trac/browser/trunk/schooltool/src/schooltool/skin/templates/table_container.pt

But please talk to Ignas (on irc, or ignas.mikalajunas at gmail.com) for 
more information.

If this code is interesting, we have a problem, as Schooltool code is 
GPL. Generalizing it and putting it in Zope's svn is thus blocked. We 
could go two routes:

* contact Mark Shuttleworth as the Zope Foundation and ask whether we 
can get this code as ZPL in the Zope repository. I can start this 
process if needed - let me know.

* talk to Ignas to get the general idea, study the code, and reimplement 
the concepts as a Zope 3 package without copying the code.

Regards,

Martijn