On Thu, Mar 11, 2010 at 08:52:11PM +0000, Jonathan Lange wrote:
For a long time, I've wanted to have a subunit output formatter for zope.testing. Subunit is a language-generic streaming protocol for test results. Once a test runner can display subunit output, it becomes much easier to write tools to process output programmatically. We want to use it in the Launchpad project for analyzing test results and for distributing our test runs across multiple machines. Other projects use it to get Hudson integration, or to glue their multi-language test suite into one result.
I like this. Incidentally, how do you integrate it with Hudson? subunit2junitxml, I assume? How exactly do you combine bin/test with subunit2junitxml -- I assume some care with respect to exit codes?
=== modified file 'src/zope/testing/testrunner/formatter.py' --- src/zope/testing/testrunner/formatter.py 2009-12-23 21:21:53 +0000 +++ src/zope/testing/testrunner/formatter.py 2010-03-11 20:29:26 +0000 ... + def __init__(self, options): + if subunit is None: + raise Exception("Requires subunit 0.0.4 or better") + if content is None: + raise Exception("Requires testtools XXX or better")
This seems a mite imprecise (unless testtools uses roman numerals for version numbering).
=== modified file 'src/zope/testing/testrunner/testrunner-leaks.txt' --- src/zope/testing/testrunner/testrunner-leaks.txt 2008-05-05 18:50:48 +0000 +++ src/zope/testing/testrunner/testrunner-leaks.txt 2010-02-20 22:23:32 +0000 @@ -16,7 +16,7 @@ >>> from zope.testing import testrunner
>>> sys.argv = 'test --layer Layer11$ --layer Layer12$ -N4 -r'.split() - >>> _ = testrunner.run(defaults) + >>> _ = testrunner.run_internal(defaults)
I'm curious about this change.
Running samplelayers.Layer11 tests: Set up samplelayers.Layer1 in 0.000 seconds. Set up samplelayers.Layer11 in 0.000 seconds. @@ -60,7 +60,7 @@ Let's look at an example test that leaks:
>>> sys.argv = 'test --tests-pattern leak -N4 -r'.split() - >>> _ = testrunner.run(defaults) + >>> _ = testrunner.run_internal(defaults) Running zope.testing.testrunner.layer.UnitTests tests:... Iteration 1 Ran 1 tests with 0 failures and 0 errors in 0.000 seconds. @@ -81,7 +81,7 @@ type (or class):
>>> sys.argv = 'test --tests-pattern leak -N5 -r -v'.split() - >>> _ = testrunner.run(defaults) + >>> _ = testrunner.run_internal(defaults) Running tests at level 1 Running zope.testing.testrunner.layer.UnitTests tests:... Iteration 1
+1 for the rest. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development