On Thu, Mar 11, 2010 at 9:17 PM, Marius Gedminas <marius@gedmin.as> wrote:
On Thu, Mar 11, 2010 at 08:52:11PM +0000, Jonathan Lange wrote:
For a long time, I've wanted to have a subunit output formatter for zope.testing. Subunit is a language-generic streaming protocol for test results. Once a test runner can display subunit output, it becomes much easier to write tools to process output programmatically. We want to use it in the Launchpad project for analyzing test results and for distributing our test runs across multiple machines. Other projects use it to get Hudson integration, or to glue their multi-language test suite into one result.
I like this.
Thanks.
Incidentally, how do you integrate it with Hudson? subunit2junitxml, I assume? How exactly do you combine bin/test with subunit2junitxml -- I assume some care with respect to exit codes?
I haven't tried myself, but from analogy with other subunit tools: ./bin/test --subunit | subunit2junitxml The formatter doesn't control exit codes, so I didn't really think about them in this patch. Should I produce a follow up?
=== modified file 'src/zope/testing/testrunner/formatter.py' --- src/zope/testing/testrunner/formatter.py 2009-12-23 21:21:53 +0000 +++ src/zope/testing/testrunner/formatter.py 2010-03-11 20:29:26 +0000 ... + def __init__(self, options): + if subunit is None: + raise Exception("Requires subunit 0.0.4 or better") + if content is None: + raise Exception("Requires testtools XXX or better")
This seems a mite imprecise (unless testtools uses roman numerals for version numbering).
Oops :) I've changed it to say 0.9.2, and subunit to say 0.0.5, which is the actual version I've tested with.
=== modified file 'src/zope/testing/testrunner/testrunner-leaks.txt' --- src/zope/testing/testrunner/testrunner-leaks.txt 2008-05-05 18:50:48 +0000 +++ src/zope/testing/testrunner/testrunner-leaks.txt 2010-02-20 22:23:32 +0000 @@ -16,7 +16,7 @@ >>> from zope.testing import testrunner
>>> sys.argv = 'test --layer Layer11$ --layer Layer12$ -N4 -r'.split() - >>> _ = testrunner.run(defaults) + >>> _ = testrunner.run_internal(defaults)
I'm curious about this change.
Sidnei explained it correctly. This particular file is only tested if you're running a Python with --with-pydebug set, so it's easy to miss when you make a change. jml