Fwd: Multi-argument append() is illegal
Fredrik Lundh has mentioned that this breaks Zope (and Medusa). If so, it's probably not too soon to begin looking into it. 1.6 is due this summer. ---------- ## Forwarded Message ## ---------- Subject: Multi-argument append() is illegal Date: Mon, 28 Feb 2000 10:41:02 -0500 From: Guido van Rossum <guido@python.org> I've noticed that there is some code out there that creates a list of tuples and uses code like list.append(a,b,c) to add the tuple (a,b,c) to the list. According to the documentation, this is illegal: append() only takes a single argument, and one should write list.append((a,b,c)). However, the actual append() implementation didn't mind, and implemented list.append(a,b,c) as list.append((a,b,c)). Many people are using this even though it's never been documented. I am going to rectify this in Python 1.6 -- people coming from other languages might well expect list.append(a, b, c) to mean the same as list.append(a); list.append(b); list.append(c), and it's always been my philosophy to make ambiguous syntax illegal rather than to pick one interpretation randomly. This message is simply a heads-up that you should be aware of this change (when 1.6 comes out, which should be before the summer). You can test your programs using the current CVS version (see www.python.org/download/cvs.html). You can also grep through your sources for a pattern like "\. *append *\(.*," -- which doesn't find every occurrence, but is a good starting point. If you have a smarter grep-like tool you may be able to write a tighter matching expression. Watch out for false hits though: some classes define their own multi-argument append()... --Guido van Rossum (home page: http://www.python.org/~guido/) -- http://www.python.org/mailman/listinfo/python-list -------------------------------------------------------
From: "Patrick Phalen" <zope@teleo.net>
Fredrik Lundh has mentioned that this breaks Zope (and Medusa).
If so, it's probably not too soon to begin looking into it. 1.6 is due this summer.
---------- ## Forwarded Message ## ---------- Subject: Multi-argument append() is illegal Date: Mon, 28 Feb 2000 10:41:02 -0500 From: Guido van Rossum <guido@python.org>
I've noticed that there is some code out there that creates a list of tuples and uses code like list.append(a,b,c) to add the tuple (a,b,c) to the list. According to the documentation, this is illegal: append() only takes a single argument, and one should write list.append((a,b,c)). However, the actual append() implementation didn't mind, and implemented list.append(a,b,c) as list.append((a,b,c)). Many people are using this even though it's never been documented.
I am going to rectify this in Python 1.6 -- people coming from other languages might well expect list.append(a, b, c) to mean the same as list.append(a); list.append(b); list.append(c), and it's always been my philosophy to make ambiguous syntax illegal rather than to pick one interpretation randomly.
This message is simply a heads-up that you should be aware of this change (when 1.6 comes out, which should be before the summer). You can test your programs using the current CVS version (see www.python.org/download/cvs.html). You can also grep through your sources for a pattern like "\. *append *\(.*," -- which doesn't find every occurrence, but is a good starting point. If you have a smarter grep-like tool you may be able to write a tighter matching expression.
Watch out for false hits though: some classes define their own multi-argument append()...
--Guido van Rossum (home page: http://www.python.org/~guido/)
Using a more advanced pattern (one that weeds out most proper appends, and takes multiple lines into account), plus some hand weeding, brings up the following list of 'illegal' tuple appends: lib\python\ExportImportXML.py(109): self._tindex.append(self._oid, here) lib\python\Shared\DC\ZRDB\RDB.py(199): parsers.append(i,parser) lib\python\Shared\DC\xml\ppml.py(278): def __setitem__(self, k, v): self._d.append(k,v) lib\python\ZODB\FileStorage.py(633): self._tappend(oid, here) lib\python\ZPublisher\cgi.py(603): r.append(name, value) utilities\FS.py(145): self._tappend(oid, here) ZServer\HTTPServer.py(285): request.channel.queue.append(self.module_name, zrequest, zresponse) ZServer\medusa\asyncore.py(84): l.append (fd, flags) Above 8 occurences are all cases of tuples being appended to a list, using the 'illegal' style syntax. Note that I still may have missed occurences, and that Python 1.6 might break other things. The change from 1.5.1 to 1.5.2 broke certain things in Zope as well, usually because of library changes. For those of you that grok POSIX regexp, here is what I used to search these out: append[[:blank:]]*\(([^[{(,]|[[:space:]])+, where [:blank:] stands for whitespace (not including newlines), and [:space:] is whitespace (including newline). This will miss aliasing of .append (app=list.append), which I hunted out with a separate regexp, and lines like: append(1 # number 1 ,2 # number 2 which were again hunted (and none found) with another simple regexp. So I am fairly sure I got em all =). I have posted these to the Collector. Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | The Open Source Web Application Server ---------------------------------------------
[Martijn Pieters, on Fri, 03 Mar 2000] :: Using a more advanced pattern (one that weeds out most proper appends, and :: takes multiple lines into account), plus some hand weeding, brings up the :: following list of 'illegal' tuple appends: [snip] Here, courtesy of Tim Peters, is checkappend.py.It might be interesting to see if it also turns up eight examples. I have seen a comment by Fredrik Lundh that it may need to be run multiple times to catch everything, but haven't drilled down to figure out why that may be true. #! /usr/bin/env python # Released to the public domain, by Tim Peters, 28 February 2000. """checkappend.py -- search for multi-argument .append() calls. Usage: specify one or more file or directory paths: checkappend [-v] file_or_dir [file_or_dir] ... Each file_or_dir is checked for multi-argument .append() calls. When a directory, all .py files in the directory, and recursively in its subdirectories, are checked. Use -v for status msgs. Use -vv for more status msgs. In the absence of -v, the only output is pairs of the form filename(linenumber): line containing the suspicious append Note that this finds multi-argument append calls regardless of whether they're attached to list objects. If a module defines a class with an append method that takes more than one argument, calls to that method will be listed. Note that this will not find multi-argument list.append calls made via a bound method object. For example, this is not caught: somelist = [] push = somelist.append push(1, 2, 3) """ __version__ = 1, 0, 0 import os import sys import string import getopt import tokenize verbose = 0 def errprint(*args): msg = string.join(args) sys.stderr.write(msg) sys.stderr.write("\n") def main(): args = sys.argv[1:] global verbose try: opts, args = getopt.getopt(sys.argv[1:], "v") except getopt.error, msg: errprint(msg + "\n\n" + __doc__) return for opt, optarg in opts: if opt == '-v': verbose = verbose + 1 if not args: errprint(__doc__) return for arg in args: check(arg) def check(file): if os.path.isdir(file) and not os.path.islink(file): if verbose: print "%s: listing directory" % `file` names = os.listdir(file) for name in names: fullname = os.path.join(file, name) if ((os.path.isdir(fullname) and not os.path.islink(fullname)) or os.path.normcase(name[-3:]) == ".py"): check(fullname) return try: f = open(file) except IOError, msg: errprint("%s: I/O Error: %s" % (`file`, str(msg))) return if verbose > 1: print "checking", `file`, "..." ok = AppendChecker(file, f).run() if verbose and ok: print "%s: Clean bill of health." % `file` [FIND_DOT, FIND_APPEND, FIND_LPAREN, FIND_COMMA, FIND_STMT] = range(5) class AppendChecker: def __init__(self, fname, file): self.fname = fname self.file = file self.state = FIND_DOT self.nerrors = 0 def run(self): try: tokenize.tokenize(self.file.readline, self.tokeneater) except tokenize.TokenError, msg: errprint("%s: Token Error: %s" % (`self.fname`, str(msg))) self.nerrors = self.nerrors + 1 return self.nerrors == 0 def tokeneater(self, type, token, start, end, line, NEWLINE=tokenize.NEWLINE, JUNK=(tokenize.COMMENT, tokenize.NL), OP=tokenize.OP, NAME=tokenize.NAME): state = self.state if type in JUNK: pass elif state is FIND_DOT: if type is OP and token == ".": state = FIND_APPEND elif state is FIND_APPEND: if type is NAME and token == "append": self.line = line self.lineno = start[0] state = FIND_LPAREN else: state = FIND_DOT elif state is FIND_LPAREN: if type is OP and token == "(": self.level = 1 state = FIND_COMMA else: state = FIND_DOT elif state is FIND_COMMA: if type is OP: if token in ("(", "{", "["): self.level = self.level + 1 elif token in (")", "}", "]"): self.level = self.level - 1 if self.level == 0: state = FIND_DOT elif token == "," and self.level == 1: self.nerrors = self.nerrors + 1 print "%s(%d):\n%s" % (self.fname, self.lineno, self.line) # don't gripe about this stmt again state = FIND_STMT elif state is FIND_STMT: if type is NEWLINE: state = FIND_DOT else: raise SystemError("unknown internal state '%s'" % `state`) self.state = state if __name__ == '__main__': main() # end of file
From: "Patrick Phalen" <zope@teleo.net>
Here, courtesy of Tim Peters, is checkappend.py.It might be interesting to see if it also turns up eight examples. I have seen a comment by Fredrik Lundh that it may need to be run multiple times to catch everything, but haven't drilled down to figure out why that may be true.
I take it he means you have to fix some of them, you run it again to find any it might have missed before you fixed them? Anyway, because the indentation didn't arrive in one piece on this end of the wire, here's a link to the original posting: http://www.deja.com/getdoc.xp?AN=591243549 Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | The Open Source Web Application Server ---------------------------------------------
From: "Patrick Phalen" <zope@teleo.net>
Here, courtesy of Tim Peters, is checkappend.py.It might be interesting to see if it also turns up eight examples.
It gets you 7, which is one more direct one then I found (my regexp must've missed the newline in that one somehow), and 2 that would be missed (as documented). The ones missed are of the bound method type, like this one: utilities\FS.py(145): self._tappend(oid, here) where self._tappend is a bound append. The one more that was found: ZServer\medusa\asyncore.py(377): tbinfo.append ( tb.tb_frame.f_code.co_filename, tb.tb_frame.f_code.co_name, str(tb.tb_lineno) ) Thanks for that pointer Patrick. Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | The Open Source Web Application Server ---------------------------------------------
participants (2)
-
Martijn Pieters -
Patrick Phalen