[Zope-dev] Re: Unit Test Failures

[Philipp von Weitershausen]

[resending this from the right address, the original posting got
swallowed by Mailman]

Tres Seaver wrote:
> Benji York wrote:
> > The Zope2 unit tests have been failing for some time on
> > buildbot.zope.com.  Looks like a Five-related problem:
> > http://buildbot.zope.org/Zope%20trunk%202.4%20Linux%20zc-buildbot/builds/109/test/0
>
> The failing test is a functional test, not a unit test;  I don't run
> them by default when I check in.

Actually, I put this particular test (functional.txt) on level=2 because
it would fail when run together with other Five tests. It passes fine
when run by itself. This test exists identically in Five 1.2 (based on
Zope 2.8) where it passes just fine when run together with other tests
and by itself. During the hot phase of making the deadline on November
1st, I decided that this wasn't a big issue and could be resolved during
the beta phase (after all, the test passes by itself). I obviously
forgot to look into it again, but then again, I still don't know what
the heck is wrong with this.

> I see a couple of problem points:  the
> biggest is the use in the output of 'checkSiteManager.html' of a repr'ed
> dict, which might render differently between different machines (or even
> the same machine across Python versions).

Well, if you look closer you find that it uses pprint.pformat which
always outputs the same on all machines (because it provides output
sorted by the dictionary key).

> This is a classic case where
> the "doctest" does *not* provide clarity over a "traditional" 'unittest'
> style test, IMNSHO.

Comparing dictionaries is not trivial in classic javiotic tests, either.
Using pprint is actually quite an elegant way, I think.

> I'm not sure what it is testing, either;

It's testing how local site managers get found during traversal and how
that affects component lookup of e.g. views (depending on which site is
traversed, you would find local components or not).

Philipp