[Zope-dev] Re: Unit Test Failures

[Tim Peters]

[Benji York]
>> The Zope2 unit tests have been failing for some time on
>> buildbot.zope.com.  Looks like a Five-related problem:
>> http://buildbot.zope.org/Zope%20trunk%202.4%20Linux%20zc-buildbot/builds/109/test/0

[Tres Seaver]
> The failing test is a functional test, not a unit test;  I don't run
> them by default when I check in.  I see a couple of problem points:  the
> biggest is the use in the output of 'checkSiteManager.html' of a repr'ed
> dict, which might render differently between different machines (or even
> the same machine across Python versions).  This is a classic case where
> the "doctest" does *not* provide clarity over a "traditional" 'unittest'
> style test, IMNSHO.

Possibly, but that clearly doesn't apply to the specific failures
we're seeing ("True" versus "False" outcomes, and getting lots more
output than expected).

> I'm not sure what it is testing, either;  CC'ing Phillip, whose
> fingerprints are on it, according the 'svn blame', for clarification.

These tests have "always" failed, and Phillip doesn't know why. 
Because they were failing, he changed them to run at level 2.  That's
not what level 2 is for, though, and the failures became "visible" to
everyone when the buildbot was changed to pass `--all` to test.py
(--all runs tests at all levels, level 2 among them ;-)).

I opened a Collector issue on this about a month ago (I always run
with `--all`, so these failures are old news to me):

    http://www.zope.org/Collectors/Zope/1947