[Zope-dev] zope.[testing.]testrunner does not handle unexpected successes correctly

[Michael Howitz]

Hi,

Python 2.7 added the concept of expectedFailures to unittest.
See http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping

When a test decorated with @unittest.expectedFailure does not raise an exception, this test is counted as success when running the tests with zope.[testing.]testrunner instead of being counted separately as 'unexpected success'.

It should be handled separately as a new result class besides failures and errors.

As it would be heavy to implement the whole concept I have a workaround which is supported by Python's unittest:

When `addUnexpectedSuccess` on zope.testrunner.runner.TestResult is None, we signal unittest that zope.testrunner ist not able to handle unexpected successes.
Although this produces a Warning it counts the unexpected success as failure instead of being ignored.

I wrote a bug report: https://bugs.launchpad.net/zope.testing/+bug/719369
and prepared a branch with the described workaround: http://svn.zope.org/zope.testrunner/branches/icemac-lp-719369

If no-one objects I'd like to merge it to the trunk and cut a new release.

zope.testing.testrunner has same problem, so I'd like to add the workaround to the 3.9 branch there, too. So it can be used in Zope 2.13.x, which still uses zope.testing.testrunner.


Yours sincerely,
-- 
Michael Howitz · mh at gocept.com · software developer
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1
Zope and Plone consulting and development