zope.testrunner test failures on windows
Hi All, I could have sworn I reported this before but couldn't find anything in the archives so apologies if I have posted this before... Anyway, running zope.testrunner's trunk's tests on windows seems to have some problems. These don't occur when running them on Linux. cheers, Chris The output:
bin\test.exe Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
Failure in test zope.testing\src\zope\testing\testrunner\testrunner- debugging-layer-setup.test Failed doctest test for testrunner-debugging-layer-setup.test File "zope.testing\src\zope\testing\testrunner\testrunner-debuggin g-layer-setup.test", line 0 ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-debugging- layer-setup.test", line 45, in testrunner-debugging-layer-setup.test Failed example: try: zope.testing.testrunner.run(['--path', dir, '-D']) finally: sys.stdin = real_stdin # doctest: +ELLIPSIS Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-debugging-layer-setup.test[10]>", line 2, in ? File "zope.testing\src\zope\testing\testrunner\__init__.py", l ine 33, in run runner.run() File "zope.testing\src\zope\testing\testrunner\runner.py", lin e 129, in run feature.global_setup() File "zope.testing\src\zope\testing\testrunner\find.py", line 343, in global_setup tests = find_tests(self.runner.options, self.runner.found_suites) File "zope.testing\src\zope\testing\testrunner\find.py", line 53, in find_tests for suite in found_suites: File "zope.testing\src\zope\testing\testrunner\find.py", line 103, in find_suites suite = StartUpFailure( File "zope.testing\src\zope\testing\testrunner\find.py", line 36, in __init__ post_mortem(exc_info) File "zope.testing\src\zope\testing\testrunner\runner.py", lin e 717, in post_mortem pdb.post_mortem(exc_info[2]) File "Python24\lib\pdb.py", line 1016, in post_mortem while t.tb_next is not None: AttributeError: 'NoneType' object has no attribute 'tb_next' A message on stderr. Please ignore (expected in test output). Failure in test zope.testing\src\zope\testing\testrunner\testrunner- coverage-win32.txt Failed doctest test for testrunner-coverage-win32.txt File "zope.testing\src\zope\testing\testrunner\testrunner-coverage -win32.txt", line 0 ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 14, in testrunner-coverage-win32.txt Failed example: ignore = testrunner.TestIgnore(WinOptions()) Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[2]>", line 1, in ? AttributeError: 'module' object has no attribute 'TestIgnore' ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 15, in testrunner-coverage-win32.txt Failed example: ignore._test_dirs Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[3]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 20, in testrunner-coverage-win32.txt Failed example: ignore.names('r:\\winproject\\src\\blah\\foo\\baz.py', 'baz') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[4]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 22, in testrunner-coverage-win32.txt Failed example: ignore.names('R:\\winproject\\src\\blah\\foo\\baz.py', 'baz') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[5]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 24, in testrunner-coverage-win32.txt Failed example: ignore.names('r:\\winproject\\src\\blah\\bar\\zab.py', 'zab') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[6]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 26, in testrunner-coverage-win32.txt Failed example: ignore.names('R:\\winproject\\src\\blah\\bar\\zab.py', 'zab') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[7]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 28, in testrunner-coverage-win32.txt Failed example: ignore.names('r:\\winproject\\src\\blah\\hello.py', 'hello') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[8]>", line 1, in ? NameError: name 'ignore' is not defined ---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 30, in testrunner-coverage-win32.txt Failed example: ignore.names('R:\\winproject\\src\\blah\\hello.py', 'hello') Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[9]>", line 1, in ? NameError: name 'ignore' is not defined Ran 183 tests with 2 failures and 0 errors in 43.094 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Tue, Aug 12, 2008 at 09:17:39PM +0100, Chris Withers wrote:
I could have sworn I reported this before but couldn't find anything in the archives so apologies if I have posted this before...
I have a feeling of deja vu about this...
Anyway, running zope.testrunner's trunk's tests on windows seems to have some problems. These don't occur when running them on Linux.
I'm vaguely interested in fixing these, but I don't do Windows.
File "zope.testing\src\zope\testing\testrunner\testrunner-debugging- layer-setup.test", line 45, in testrunner-debugging-layer-setup.test ... File "zope.testing\src\zope\testing\testrunner\find.py", line 103, in find_suites suite = StartUpFailure(
Let's look at this statement closer: try: module = import_name(module_name) except KeyboardInterrupt: raise except: --> suite = StartUpFailure( options, module_name, sys.exc_info()[:2] + (sys.exc_info()[2].tb_next.tb_next,), ) It would appear that the traceback is shallower than the code expects. I don't know why it is trying to strip the first two frames of the stack, either.
File "zope.testing\src\zope\testing\testrunner\find.py", line 36, in __init__ post_mortem(exc_info) File "zope.testing\src\zope\testing\testrunner\runner.py", lin e 717, in post_mortem pdb.post_mortem(exc_info[2]) File "Python24\lib\pdb.py", line 1016, in post_mortem while t.tb_next is not None:
Apparently the traceback was exactly two frames deep, and with those stripped, the code passed None to pdb.post_mortem(). It would be good if someone who has Windows and Linux could compare the tracebacks in question. I'd also suggest extracting the stack frame stripping into a function that would be a bit more defensive: def strip_tb_frames((exc_type, exc_value, tb), count): while tb.tb_next is not None and count > 0: tb = tb.tb_next count -= 1 return (exc_type, exc_value, tb) and call it like this suite = StartUpFailure( options, module_name, strip_tb_frames(sys.exc_info(), 2) ) (Why oh why does Guido want to take away my beloved implicit argument tuple unpacking in Python 3.0?)
AttributeError: 'NoneType' object has no attribute 'tb_next'
Next error:
Failure in test zope.testing\src\zope\testing\testrunner\testrunner- coverage-win32.txt Failed doctest test for testrunner-coverage-win32.txt File "zope.testing\src\zope\testing\testrunner\testrunner-coverage -win32.txt", line 0
The name of this test suggests it's only run under Win32, so any refactoring done on Linux is likely to break it without anyone noticing.
---------------------------------------------------------------------- File "zope.testing\src\zope\testing\testrunner\testrunner-coverage-w in32.txt", line 14, in testrunner-coverage-win32.txt Failed example: ignore = testrunner.TestIgnore(WinOptions()) Exception raised: Traceback (most recent call last): File "zope.testing\src\zope\testing\doctest.py", line 1356, in __run compileflags, 1) in test.globs File "<doctest testrunner-coverage-win32.txt[2]>", line 1, in ? AttributeError: 'module' object has no attribute 'TestIgnore'
TestIgnore moved from zope.testing.testrunner to zope.testing.testrunner.coverage. It should be simple to fix the test.
Ran 183 tests with 2 failures and 0 errors in 43.094 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
Marius Gedminas -- An NT server can be run by an idiot, and usually is.
Marius Gedminas wrote:
On Tue, Aug 12, 2008 at 09:17:39PM +0100, Chris Withers wrote:
I could have sworn I reported this before but couldn't find anything in the archives so apologies if I have posted this before...
I have a feeling of deja vu about this...
Indeed ;-)
File "zope.testing\src\zope\testing\testrunner\testrunner-debugging- layer-setup.test", line 45, in testrunner-debugging-layer-setup.test ... File "zope.testing\src\zope\testing\testrunner\find.py", line 103, in find_suites suite = StartUpFailure(
Let's look at this statement closer:
Wrong statement ;-) The offending chunk was further down in the file, and the reasons for the failure are manyfold :-( 1. StartUpFailure there gets called with the exception traceback explicitly set to None. That means if you enable postmortem debugging and a StartUpFailure gets created without full exc_info, you get the AttributeError on tb_next. I've added a doctest and code to StartUpFailure to raise a more explicit error in this case. 2. It appears the traceback is explicitly filtered out of exc_info in the second place StartUpFailures are instantiated purely to make the output a bit nicer. What would people feel about providing the traceback here to enable postmortem debugging to work properly? I've started adding test cases for this to testrunner-edge-cases.txt but want to know what to do before finishing them off :-S 3. testrunner-debugging-layer-setup.test appears to be running the testrunner with insuffient parameters to only find the tests it has set up. In my case, it's for some reason trying to find tests within eggs in my base python installation (yay! for setuptools for breaking my base setup, but that's another issue entirely ;-)) What parameters do we need to change in that test to make it work as intended? The module where find_suites is failing to find a test_suite function in question is: <module 'tests' from 'Python24/lib/site-packages/mailinglogger-3.1.0-py2.4.egg/tests/__init__.pyc'> 4. I don't understand why find_suites would be unhappy to not find a test_suite method in a __init__.py. 'tests' is a package in mailinglogger. That said, looking at that 'egg', it looks pretty broken so the tests package is probably ending up actually being on sys.path. Still, why would find_tests try and find tests in it? Unfortunately, when Christian did his refactoring he did so without doing an svn-cp-then-edit, so I don't even know who wrote the offending code :-S
File "zope.testing\src\zope\testing\testrunner\find.py", line 36, in __init__ post_mortem(exc_info) File "zope.testing\src\zope\testing\testrunner\runner.py", lin e 717, in post_mortem pdb.post_mortem(exc_info[2]) File "Python24\lib\pdb.py", line 1016, in post_mortem while t.tb_next is not None:
Apparently the traceback was exactly two frames deep, and with those stripped, the code passed None to pdb.post_mortem().
Yes, although I'll note that the code has changed since I ran these, as Michael Howitz added a test for StartUpFailure at my request.
I'd also suggest extracting the stack frame stripping into a function that would be a bit more defensive:
def strip_tb_frames((exc_type, exc_value, tb), count): while tb.tb_next is not None and count > 0: tb = tb.tb_next count -= 1 return (exc_type, exc_value, tb)
and call it like this
suite = StartUpFailure( options, module_name, strip_tb_frames(sys.exc_info(), 2) )
I have a sneaking suspiction the stripping of two levels is purely to remove the frames that are actually inside find.py. If this is the case, a comment would likely suffice as to why it's there...
(Why oh why does Guido want to take away my beloved implicit argument tuple unpacking in Python 3.0?)
AttributeError: 'NoneType' object has no attribute 'tb_next'
Next error: TestIgnore moved from zope.testing.testrunner to zope.testing.testrunner.coverage. It should be simple to fix the test.
Well, correcting the import revealed some other massaging that needed to be done but this is now fixed :-) *multi-sigh* Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote at 2008-8-16 18:22 +0100:
... 2. It appears the traceback is explicitly filtered out of exc_info in the second place StartUpFailures are instantiated purely to make the output a bit nicer. What would people feel about providing the traceback here to enable postmortem debugging to work properly?
+1 -- Dieter
Dieter Maurer wrote:
Chris Withers wrote at 2008-8-16 18:22 +0100:
... 2. It appears the traceback is explicitly filtered out of exc_info in the second place StartUpFailures are instantiated purely to make the output a bit nicer. What would people feel about providing the traceback here to enable postmortem debugging to work properly?
+1
Does anyone else have any views on this? If I do add this, it will mean that longer tracebacks show up where currently only the name and type of exception show up. Are people okay with this? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (3)
-
Chris Withers -
Dieter Maurer -
Marius Gedminas