Re: [Zope] dtml-in sort by generic function
Oleg Broytmann writes:
... controling sorting in "dtml-in" ... .... but a colleague of mine, Dennis Otkidach suggested more general approach - to make a patch that will allow to pass a generic function to sort(), something like <dtml-in seq sort=surname sort_func=foobar> where sort_func points to a Python function (like locale.strcoll), or Python Script function, or such.
Any opinion? I see a minor problem here - if I want just locale-aware sort, I would write <dtml-in seq sort=surname sort_func=_.locale.strcoll>, A very good idea!
However, you should note: * "dtml-in" does not sort single values, but tuples ( sort_values, client), where "sort_values" is either a single value or a list of values and "client" is the current element. Your sort function must handle this case * "dtml-in" supports multiple sort keys. I would expect to have a comparison function for each sort key, with the global comparison function a composition of that for the sort keys. Thus, you can combine a locale sensitive sort with (e.g.) a date sort. Dieter
Hi! Thank you for stepping in! On Tue, 13 Mar 2001, Dieter Maurer wrote:
but a colleague of mine, Dennis Otkidach suggested more general approach - to make a patch that will allow to pass a generic function to sort(), A very good idea!
However, you should note:
* "dtml-in" does not sort single values, but tuples ( sort_values, client), where "sort_values" is either a single value or a list of values and "client" is the current element.
Your sort function must handle this case
I know it! From time to time I scan and rescan Zope sources just for better understanding and finding of hidden treasures (there are many :).
* "dtml-in" supports multiple sort keys. I would expect to have a comparison function for each sort key, with the global comparison function a composition of that for the sort keys.
Thus, you can combine a locale sensitive sort with (e.g.) a date sort.
But of course. Or simulating SQL "ORDER BY date DESC, title ASC" :) But I do not see a way to do it in generic fashion. I can write a specific comparison function for every specific case, but how I can combine few comparison functions into one in a generic way? Imagine a have a string comparison str_cmp, and date comparison date_cmp functions. Now I need to pass to dtml-in a combined function. Should it be a Python Script that just calls str_cmp and date_cmp in turn, something like result = str_cmp(s1[0], s2[0]) if result: return result # if not equal - order by first key result = date_cmp(s1[1], s2[1]) # else order by second key return result Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann writes:
Thank you for stepping in! I like good ideas. They must be supported...
* "dtml-in" supports multiple sort keys. I would expect to have a comparison function for each sort key, with the global comparison function a composition of that for the sort keys.
Thus, you can combine a locale sensitive sort with (e.g.) a date sort.
But of course. Or simulating SQL "ORDER BY date DESC, title ASC" :)
But I do not see a way to do it in generic fashion. I can write a specific comparison function for every specific case, but how I can combine few comparison functions into one in a generic way? Imagine a have a string comparison str_cmp, and date comparison date_cmp functions. Now I need to pass to dtml-in a combined function. Should it be a Python Script that just calls str_cmp and date_cmp in turn, something like I am not yet clear about a good syntax. It should be something, that pairs the attribute with the sorting function essential for this attribute: something like:
sort="attr1[cmp1],attr2,attr3[cmp3],...." This should mean, use "cmp1" for "attr1", the default comparison for "attr2" (i.e. cmp) and "cmp3" for "attr3" (and so on). Each "cmp" is looked up in the namespace and if not found there, in a standard set of comparison function, such that it is easy to get "locale_asc", "locale_desc", "case_insensitive_asc", .... This way, you get a tuple of comparison functions of the same length as the tuple of values. Now we define: def lexicographicCompare(values1,values2,functions): '''*values1*, *values2* and *functions* are all tuples of the same length with "functions[i]" capable of comparing "value1[i]" and "values2[i]". The result is the lexicographic comparison of the tuples.''' for v1,v2,cmp in map(None,value1,value2,functions): c= cmp(v1,v2) # maybe we should do something, if this raises an exception if c: return c return 0 Dieter
On Wed, 14 Mar 2001, Dieter Maurer wrote:
I am not yet clear about a good syntax. It should be something, that pairs the attribute with the sorting function essential for this attribute: something like:
sort="attr1[cmp1],attr2,attr3[cmp3],...."
Looks good, although a bit too complex.
This should mean, use "cmp1" for "attr1", the default comparison for "attr2" (i.e. cmp) and "cmp3" for "attr3" (and so on).
Each "cmp" is looked up in the namespace and if not found there, in a standard set of comparison function, such that it is easy to get "locale_asc", "locale_desc", "case_insensitive_asc", ....
I prefer not to have more specail keywords. Better, let's predefine some functions (case_insensitive_asc, case_insensitive_desc), and some functions aonly if locale is already imported (sys.modules.has_key("locale")).
This way, you get a tuple of comparison functions of the same length as the tuple of values.
Now we define: def lexicographicCompare(values1,values2,functions): '''*values1*, *values2* and *functions* are all tuples of the same length with "functions[i]" capable of comparing "value1[i]" and "values2[i]". The result is the lexicographic comparison of the tuples.''' for v1,v2,cmp in map(None,value1,value2,functions): c= cmp(v1,v2) # maybe we should do something, if this raises an exception if c: return c return 0
That's quite good! Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hello! I returned to the idea after spending some time with bugs in b2; all was fixed in b3 at last. I am ready to start coding and patching dtml-in. On Wed, 14 Mar 2001, Dieter Maurer wrote:
I am not yet clear about a good syntax. It should be something, that pairs the attribute with the sorting function essential for this attribute: something like:
sort="attr1[cmp1],attr2,attr3[cmp3],...."
This should mean, use "cmp1" for "attr1", the default comparison for "attr2" (i.e. cmp) and "cmp3" for "attr3" (and so on).
Each "cmp" is looked up in the namespace and if not found there, in a standard set of comparison function, such that it is easy to get "locale_asc", "locale_desc", "case_insensitive_asc", ....
As I already said, this looks good... but I don't want to have a predefined set of names. I want these "locale_asc"/"locale_desc" to be the real functions. And I don't want to have 2 almost identical functions (asc/desc) for every comparison function. So I am thinking about syntax like this: sort="attr1/cmp1,attr2,attr3/cmp3/desc,...." That is, attribute/function/direction. Default direction is ASC, of course. It would be easy to modify the generic function you've provided:
def lexicographicCompare(values1,values2,functions): '''*values1*, *values2* and *functions* are all tuples of the same length with "functions[i]" capable of comparing "value1[i]" and "values2[i]". The result is the lexicographic comparison of the tuples.''' for v1,v2,cmp in map(None,value1,value2,functions): c= cmp(v1,v2) # maybe we should do something, if this raises an exception if c: return c return 0
Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann writes:
On Wed, 14 Mar 2001, Dieter Maurer wrote:
I am not yet clear about a good syntax. It should be something, that pairs the attribute with the sorting function essential for this attribute: something like:
sort="attr1[cmp1],attr2,attr3[cmp3],...."
This should mean, use "cmp1" for "attr1", the default comparison for "attr2" (i.e. cmp) and "cmp3" for "attr3" (and so on).
Each "cmp" is looked up in the namespace and if not found there, in a standard set of comparison function, such that it is easy to get "locale_asc", "locale_desc", "case_insensitive_asc", ....
As I already said, this looks good... but I don't want to have a predefined set of names. Sad, because it would be difficult for non-Python programmers to define these functions.
What I suggest is an approach similar to the custom formats: there is a predefined set of well known and often used sort function (such as "locale", "case_insensitive", ...) *BUT* a user can use a function of his own, too, if he likes.
I want these "locale_asc"/"locale_desc" to be the real functions. And I don't want to have 2 almost identical functions (asc/desc) for every comparison function. So I am thinking about syntax like this:
sort="attr1/cmp1,attr2,attr3/cmp3/desc,...." To separate the direction is a good idea. I like it. I am not so sure about the '/' separators. I expect, we will soon get "/" separated object access paths. Then, the use of "/" for a completely different purpose may be confusing. Not sure, that my "[...]" is better in this respect...
Dieter
"DM" == Dieter Maurer <dieter@handshake.de> writes:
DM> To separate the direction is a good idea. I like it. I am not DM> so sure about the '/' separators. I expect, we will soon get DM> "/" separated object access paths. Then, the use of "/" for a DM> completely different purpose may be confusing. Not sure, that DM> my "[...]" is better in this respect... Zieve does this with ':' sorta like name="foo:int" Another trick from Zieve is the use of comparison classes: __doc__=''' Comparison class for sorting object based on runtime attributes... Steve Spicklemire Silicon Prairie Ventures Inc. steve@spvi.com ''' __version__='$Revision: 1.1.1.1 $'[11:-2] import string import traceback import sys class compClass: # # The idea here is to provide generic sorting... # def __init__(self, sortAttrs = None, raiseExceptions = 0, theClass = None, emptyLast = 1 ): """ set up sort attributes and type conversions....""" self.sortAttrs = [] self.raiseExceptions = raiseExceptions self.emptyLast = emptyLast if sortAttrs: for theAttr in sortAttrs: unbound = 1 if string.find(theAttr,':') != -1: theAttr, theType = string.split(theAttr,':') if self.typeFuncs.has_key(theType): theTypeFunc = self.typeFuncs[theType] elif hasattr(self, theType) and callable(getattr(self, theType)): theTypeFunc = getattr(self, theType) elif theClass and hasattr(theClass, theType) and callable(getattr(theClass, theType)): unbound = 0 theTypeFunc = getattr(theClass, theType) else: theTypeFunc = None else: theTypeFunc = None self.sortAttrs.append((theAttr, theTypeFunc, unbound)) def toInt(self, thing): result = 0 if type(thing) == type(''): try: result = string.atoi(thing) except: if self.raiseExceptions: raise else: result = thing return result def toChar(self, thing): result = '' if type(thing) != type(''): result = str(thing) else: result = thing return result def doComp(self, a, b): if self.emptyLast: # force empty strings last... if type(a) == type('') and type(b) == type(''): if len(a) == 0 and len(b) != 0: return 1 elif len(b) == 0 and len(a) != 0: return -1 if a<b: return -1 if a==b: return 0 return 1 def __call__(self, thingA, thingB): if not self.sortAttrs: return self.doComp(thingA, thingB) result = 0 for theAttr, theFunc, unbound in self.sortAttrs: if hasattr(thingA, theAttr) and hasattr(thingB, theAttr): a = getattr(thingA, theAttr) b = getattr(thingB, theAttr) if theFunc is not None: if unbound: [a,b] = map(lambda o,s=self,t=theFunc:apply(t,(s,o)),[a,b]) # map type conversion onto elements else: [a,b] = map(lambda o,s=self,t=theFunc:apply(t,(o,)),[a,b]) # map type conversion onto elements result = self.doComp(a,b) if result != 0: break return result typeFuncs = { 'int': toInt, 'char': toChar } # type conversion map.... def test(): # # test this sort thingy on some random class.... # class foo: a = 0 b = 0 first = '' def __repr__(self): return '( a->%s, b->%s, first->%s, )[%s]' % ( `self.a`, `self.b`, self.first, id(self)) x = foo() y = foo() z = foo() x.a = '30' x.b = 10 x.first = 'c' y.a = '30' y.b = 100 y.first = 'D' z.a = '200' z.b = 10 z.first = 'E' ccab = compClass(['a:int','b:char']) ccba = compClass(['b:int','a:char']) ccup = compClass(['first:toUpper']) ccabr = compClass(['a:int','b:char'], 1) cc = compClass() list1 = [x,y,z] list2 = ['200','30','300'] list1.sort(ccab) print list1 list1.sort(ccba) print list1 list2.sort(cc) print list2 list1.sort(ccup) print list1 z.a = 'foo' try: list1.sort(ccabr) except ValueError: print "we got the expected exception" gotit = 0 try: list1.sort(ccab) except: gotit = 1 if gotit: print "dang we got the exception we wanted to miss.." else: print "OK.. no exception seen..." if __name__ == '__main__': test()
On Tue, 27 Mar 2001, Steve Spicklemire wrote:
Zieve does this with ':' sorta like name="foo:int"
I am considering this, too.
Another trick from Zieve is the use of comparison classes:
Thank you for the code. I used similar class in my other project. Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
On Tue, 27 Mar 2001, Dieter Maurer wrote:
Each "cmp" is looked up in the namespace and if not found there, in a standard set of comparison function, such that it is easy to get "locale_asc", "locale_desc", "case_insensitive_asc", ....
As I already said, this looks good... but I don't want to have a predefined set of names. Sad, because it would be difficult for non-Python programmers to define these functions.
Aha, I see the point!
What I suggest is an approach similar to the custom formats:
there is a predefined set of well known and often used sort function (such as "locale", "case_insensitive", ...) *BUT* a user can use a function of his own, too, if he likes.
Ok. Do you see any non-cmp functions beynd these "locale" and case_insensitive? :) I can't work out a good set of predefined functions. All those string, date, numeric comparisons - they are just cmp and nothing more.
real functions. And I don't want to have 2 almost identical functions (asc/desc) for every comparison function. So I am thinking about syntax like this:
sort="attr1/cmp1,attr2,attr3/cmp3/desc,...." To separate the direction is a good idea. I like it. I am not so sure about the '/' separators. I expect, we will soon get "/" separated object access paths.
Not in DTML - in ZPT, but it is different beast :)
Then, the use of "/" for a completely different purpose may be confusing. Not sure, that my "[...]" is better in this respect...
Hmm... sort="attr1[cmp1],attr2,attr3[cmp3:desc],...." or ...",attr3[cmp3/desc],..." or ...",attr3[cmp3,desc],..." Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann writes:
Ok. Do you see any non-cmp functions beynd these "locale" and case_insensitive? :) I can't work out a good set of predefined functions. All those string, date, numeric comparisons - they are just cmp and nothing more. Maybe, you already found the most essential ones. Of course, it would be easy to add further ones, when we find some more interesting candidates.
BTW, are all "locale" comparison functions case insensitive? Unlike most other people, I like case sensitive comparisons. Dieter
On Wed, 28 Mar 2001, Dieter Maurer wrote:
Ok. Do you see any non-cmp functions beynd these "locale" and case_insensitive? :) I can't work out a good set of predefined functions. All those string, date, numeric comparisons - they are just cmp and nothing more. Maybe, you already found the most essential ones.
I think so :)
Of course, it would be easy to add further ones, when we find some more interesting candidates.
But of course :)
BTW, are all "locale" comparison functions case insensitive? Unlike most other people, I like case sensitive comparisons.
What do you mean? locale.strcoll? I beleive it IS case sensitive. import locale locale.setlocale(locale.LC_ALL, "") print locale.strcoll("aB", "Ab") print locale.strcoll("Ab", "aB") Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hello! On Tue, 27 Mar 2001, Dieter Maurer wrote:
sort="attr1/cmp1,attr2,attr3/cmp3/desc,...."
I did it! The patch is simple enough, to my surprise. I wrote a small test suite (7 templates) to be run from command line. The test passed both under Python 1.5.2 and 2.0. Anyone wants to see the patch and tests while I am testing patched Zope? Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Hello!
sort="attr1/cmp1,attr2,attr3/cmp3/desc,...."
I tested the patch with Zope, and it works perfectly. I put the patch and test suite into Collector: http://classic.zope.org:8080/Collector/2120/view Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann writes:
sort="attr1/cmp1,attr2,attr3/cmp3/desc,...."
I tested the patch with Zope, and it works perfectly.
I put the patch and test suite into Collector: http://classic.zope.org:8080/Collector/2120/view Great!
When I saw your announcement a crazy idea came to my mind: What, if your code would be made available as an External Method. Then, you could use it independantly from "dtml-in" for sophisticated sorting of sequences. We all know how low it may take before new functionality is incorporated into Zope (--> Tino's dtml-in patch). Having it outside the "dtml-in" makes out less dependant from DC. Dieter
On Mon, 2 Apr 2001, Dieter Maurer wrote:
When I saw your announcement a crazy idea came to my mind:
What, if your code would be made available as an External Method. Then, you could use it independantly from "dtml-in" for sophisticated sorting of sequences.
Something like <dtml-in "genericSort(mySequence, sort="weight/cmp/desci,title/locale")"> ??? Oleg. ---- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
participants (4)
-
Dieter Maurer -
Oleg Broytmann -
Oleg Broytmann -
Steve Spicklemire