Tres Seaver wrote:

I see a couple of problem points: the biggest is the use in the output of 'checkSiteManager.html' of a repr'ed dict, which might render differently between different machines (or even the same machine across Python versions). This is a classic case where the "doctest" does *not* provide clarity over a "traditional" 'unittest' style test, IMNSHO.

It is often clearer to show the contents of a dict than making an assertion. In those cases where it's not, you can make a unittest style assertion, as in:

...

...

thisdict == thatdict True

This is really just a matter of knowing how to write the test. Generally, when you want to show a dict sample, the way to do it is with:

...

...

from zope.testing/doctestunit import pprint pprint(thisdict)

This formats the dictionary nicely and, most importantly, sorts the items before displaying it. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

[Benji York]

...

The Zope2 unit tests have been failing for some time on buildbot.zope.com. Looks like a Five-related problem: http://buildbot.zope.org/Zope%20trunk%202.4%20Linux%20zc-buildbot/builds/109...

[Tres Seaver]

The failing test is a functional test, not a unit test; I don't run them by default when I check in. I see a couple of problem points: the biggest is the use in the output of 'checkSiteManager.html' of a repr'ed dict, which might render differently between different machines (or even the same machine across Python versions). This is a classic case where the "doctest" does *not* provide clarity over a "traditional" 'unittest' style test, IMNSHO.

Possibly, but that clearly doesn't apply to the specific failures we're seeing ("True" versus "False" outcomes, and getting lots more output than expected).

I'm not sure what it is testing, either; CC'ing Phillip, whose fingerprints are on it, according the 'svn blame', for clarification.

These tests have "always" failed, and Phillip doesn't know why. Because they were failing, he changed them to run at level 2. That's not what level 2 is for, though, and the failures became "visible" to everyone when the buildbot was changed to pass `--all` to test.py (--all runs tests at all levels, level 2 among them ;-)). I opened a Collector issue on this about a month ago (I always run with `--all`, so these failures are old news to me): http://www.zope.org/Collectors/Zope/1947

[Tim]

...

I opened a Collector issue on this about a month ago (I always run with `--all`, so these failures are old news to me):

http://www.zope.org/Collectors/Zope/1947

[Philipp]

Rereading that issue again, it's totally surprising to see that there's no failure on Windows, which makes its source even harder to debug.

No, these tests fail on Windows too. In fact, the problem was orginally reported against Windows: see entry #2 in: http://www.zope.org/Collectors/Zope/1931 What does not fail on Windows is the specific variation you asked me to try in your comment #2 on issue 1947: When you run it by itself, it passes, though. Maybe you can confirm that. In my comment #3 on issue 1947, I confirmed that: when I ran the functional.txt test _by itself_ on Windows, it passed.

I wouldn't know where to start (having tried to debug this problem in the past). Anyone got an idea?

For a start, disabuse yourself of the illusion that it acts differently on Windows than on Linux ;-)

[resending this from the right address, the original posting got swallowed by Mailman] Tres Seaver wrote:

Benji York wrote:

...

The Zope2 unit tests have been failing for some time on buildbot.zope.com. Looks like a Five-related problem: http://buildbot.zope.org/Zope%20trunk%202.4%20Linux%20zc-buildbot/builds/109...

The failing test is a functional test, not a unit test; I don't run them by default when I check in.

Actually, I put this particular test (functional.txt) on level=2 because it would fail when run together with other Five tests. It passes fine when run by itself. This test exists identically in Five 1.2 (based on Zope 2.8) where it passes just fine when run together with other tests and by itself. During the hot phase of making the deadline on November 1st, I decided that this wasn't a big issue and could be resolved during the beta phase (after all, the test passes by itself). I obviously forgot to look into it again, but then again, I still don't know what the heck is wrong with this.

I see a couple of problem points: the biggest is the use in the output of 'checkSiteManager.html' of a repr'ed dict, which might render differently between different machines (or even the same machine across Python versions).

Well, if you look closer you find that it uses pprint.pformat which always outputs the same on all machines (because it provides output sorted by the dictionary key).

This is a classic case where the "doctest" does *not* provide clarity over a "traditional" 'unittest' style test, IMNSHO.

Comparing dictionaries is not trivial in classic javiotic tests, either. Using pprint is actually quite an elegant way, I think.

I'm not sure what it is testing, either;

It's testing how local site managers get found during traversal and how that affects component lookup of e.g. views (depending on which site is traversed, you would find local components or not). Philipp

Jim Fulton wrote:

Obviously, some other test isn't cleaning up after itself.

Yes, that was obvious to me too. It was confusing that the same test would pass on Five 1.2, though, and I couldn't find any obvious differences. As trial and error usually takes time, I left the issue to be resolved at a later time. And then, as I said, I just forgot, my bad. I'm investigating now, but no promises that I'll produce results soon. I won't forget it this time, though :). Philipp ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.