[Zope-Coders] Regular Zope tests
Matthew T. Kromer
matt@zope.com
Mon, 22 Oct 2001 17:14:26 -0400
Shane Hathaway wrote:
> Matthew T. Kromer wrote:
>
>> Shane Hathaway wrote:
>>
>>> I got excited about the number of unit tests Zope now passes (937
>>> out of 939), so I decided to document it by beginning automatic
>>> posting of test results to zope-coders@zope.org. It will post at 10
>>> PM on weekdays, but only when any test results change.
>>>
>>> Enjoy!
>>>
>> I assert that all failed cases should post nightly, or nights prior
>> to work days (ie Sunday - Thursday).
>
>
>
> Right now no one is assigned to fix the last two test cases. Maybe
> once they are fixed we should switch to always mailing failures.
And if you never report failures, there is a lower incentive to assign
people to fix them. The point is to have it be a nusiance. I'd even
endorse a "so-and-so broke the build" mailing if there was a change
attributable to a person.
>> An OK test run should affirmatively mail that NNN/NNN test cases
>> successfully ran. I'm OK with the thing shutting up after some
>> number of days that everything is good, but I would still like to see
>> weekly reminders that things are good (which also means that the
>> auto-tester is good).
>
> Steve suggested that we might also provide a web interface. I think
> that would fulfill the need to find out the current state of things.
The point is to be a daily affirmation of the state of the tree. If I
wanted to know if the tests ran succesfully whenever-I-was-curious, I
could pull my own tree and run the tests.
>> I'd like to see test case mailings for both the current stable and
>> development branches.
>
>
> Currently the branches fail a *lot* of the tests, mainly just because
> the test scripts need to be updated. Once they are sorted out, we can
> auto-test them too.
>
> Shane
See incentive above. The point here is not to sugar coat the test
suites. If you would prefer, I'll generate the test mailing instead. I
started doing that earlier but never hooked up a mailer.
As a developer, you might only care if things are broken. As a
packager, I want to know when things are good. I want someone to tell
me, daily, that things are good. I'm amenable to being flexible about
unpublished code (e.g. the trunk) but I still think its more likely that
bad test cases get fixed if they're announced frequently, rather than
announced once and then set to a "broken and forgotten" state.