[Zope-dev] zope.testrunner test failures, remaining ones not windows specific ; -)

[Chris Withers]

Marius Gedminas wrote:
> On Tue, Aug 12, 2008 at 09:17:39PM +0100, Chris Withers wrote:
>> I could have sworn I reported this before but couldn't find anything in 
>> the archives so apologies if I have posted this before...
> 
> I have a feeling of deja vu about this...

Indeed ;-)

>> File "zope.testing\src\zope\testing\testrunner\testrunner-debugging-
>> layer-setup.test", line 45, in testrunner-debugging-layer-setup.test
> ...
>>        File "zope.testing\src\zope\testing\testrunner\find.py", line
>> 103, in find_suites
>>          suite = StartUpFailure(
> 
> Let's look at this statement closer:

Wrong statement ;-)

The offending chunk was further down in the file, and the reasons for 
the failure are manyfold :-(

1. StartUpFailure there gets called with the exception traceback 
explicitly set to None. That means if you enable postmortem debugging 
and a StartUpFailure gets created without full exc_info, you get the 
AttributeError on tb_next. I've added a doctest and code to 
StartUpFailure to raise a more explicit error in this case.

2. It appears the traceback is explicitly filtered out of exc_info in 
the second place StartUpFailures are instantiated purely to make the 
output a bit nicer. What would people feel about providing the traceback 
here to enable postmortem debugging to work properly? I've started 
adding test cases for this to testrunner-edge-cases.txt but want to know 
what to do before finishing them off :-S

3. testrunner-debugging-layer-setup.test appears to be running the 
testrunner with insuffient parameters to only find the tests it has set 
up. In my case, it's for some reason trying to find tests within eggs in 
my base python installation (yay! for setuptools for breaking my base 
setup, but that's another issue entirely ;-))
What parameters do we need to change in that test to make it work as 
intended?
The module where find_suites is failing to find a test_suite function in 
question is:
<module 'tests' from 
'Python24/lib/site-packages/mailinglogger-3.1.0-py2.4.egg/tests/__init__.pyc'>

4. I don't understand why find_suites would be unhappy to not find a 
test_suite method in a __init__.py. 'tests' is a package in 
mailinglogger. That said, looking at that 'egg', it looks pretty broken 
so the tests package is probably ending up actually being on sys.path. 
Still, why would find_tests try and find tests in it?

Unfortunately, when Christian did his refactoring he did so without 
doing an svn-cp-then-edit, so I don't even know who wrote the offending 
code :-S

>>        File "zope.testing\src\zope\testing\testrunner\find.py", line
>> 36, in __init__
>>          post_mortem(exc_info)
>>        File "zope.testing\src\zope\testing\testrunner\runner.py", lin
>> e 717, in post_mortem
>>          pdb.post_mortem(exc_info[2])
>>        File "Python24\lib\pdb.py", line 1016, in post_mortem
>>          while t.tb_next is not None:
> 
> Apparently the traceback was exactly two frames deep, and with those stripped,
> the code passed None to pdb.post_mortem().

Yes, although I'll note that the code has changed since I ran these, as 
Michael Howitz added a test for StartUpFailure at my request.

> I'd also suggest extracting the stack frame stripping into a function
> that would be a bit more defensive:
> 
>     def strip_tb_frames((exc_type, exc_value, tb), count):
>         while tb.tb_next is not None and count > 0:
>             tb = tb.tb_next
>             count -= 1
>         return (exc_type, exc_value, tb)
> 
> and call it like this
> 
>                     suite = StartUpFailure(
>                         options, module_name,
>                         strip_tb_frames(sys.exc_info(), 2)
>                         )

I have a sneaking suspiction the stripping of two levels is purely to 
remove the frames that are actually inside find.py.
If this is the case, a comment would likely suffice as to why it's there...
> 
> (Why oh why does Guido want to take away my beloved implicit argument
> tuple unpacking in Python 3.0?)
> 
>>      AttributeError: 'NoneType' object has no attribute 'tb_next'
>>
> 
> Next error:
> TestIgnore moved from zope.testing.testrunner to
> zope.testing.testrunner.coverage.  It should be simple to fix the test.

Well, correcting the import revealed some other massaging that needed to 
be done but this is now fixed :-)

*multi-sigh*

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk