zope.[testing.]testrunner does not handle unexpected successes correctly
Hi, Python 2.7 added the concept of expectedFailures to unittest. See http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping When a test decorated with @unittest.expectedFailure does not raise an exception, this test is counted as success when running the tests with zope.[testing.]testrunner instead of being counted separately as 'unexpected success'. It should be handled separately as a new result class besides failures and errors. As it would be heavy to implement the whole concept I have a workaround which is supported by Python's unittest: When `addUnexpectedSuccess` on zope.testrunner.runner.TestResult is None, we signal unittest that zope.testrunner ist not able to handle unexpected successes. Although this produces a Warning it counts the unexpected success as failure instead of being ignored. I wrote a bug report: https://bugs.launchpad.net/zope.testing/+bug/719369 and prepared a branch with the described workaround: http://svn.zope.org/zope.testrunner/branches/icemac-lp-719369 If no-one objects I'd like to merge it to the trunk and cut a new release. zope.testing.testrunner has same problem, so I'd like to add the workaround to the 3.9 branch there, too. So it can be used in Zope 2.13.x, which still uses zope.testing.testrunner. Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
Am 16.02.2011 um 10:36 schrieb Michael Howitz:
Hi,
Python 2.7 added the concept of expectedFailures to unittest. See http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping
When a test decorated with @unittest.expectedFailure does not raise an exception, this test is counted as success when running the tests with zope.[testing.]testrunner instead of being counted separately as 'unexpected success'.
It should be handled separately as a new result class besides failures and errors.
As it would be heavy to implement the whole concept I have a workaround which is supported by Python's unittest:
When `addUnexpectedSuccess` on zope.testrunner.runner.TestResult is None, we signal unittest that zope.testrunner ist not able to handle unexpected successes. Although this produces a Warning it counts the unexpected success as failure instead of being ignored.
I wrote a bug report: https://bugs.launchpad.net/zope.testing/+bug/719369 and prepared a branch with the described workaround: http://svn.zope.org/zope.testrunner/branches/icemac-lp-719369
If no-one objects I'd like to merge it to the trunk and cut a new release.
Hi, as no-one objected, I merged the workaround to the trunk and released zope.testrunner 4.0.1. I adapted the patch to zope.testing.testrunner and released there as 3.9.6. Yours sincerely, -- Michael Howitz · mh@gocept.com · software developer gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development
participants (1)
-
Michael Howitz