[Zope3-dev] The Zope Software Certification Program and Common Repository Proposal

[Benji York]

Lennart Regebro wrote:
 > Thanks for the answer. I only have one remaining comment, then, about
 >  testing:
 >
 > On 2/21/06, Stephan Richter <srichter at cosmos.phy.tufts.edu> wrote:
 >
 >> Why is it random? It is taken straight from the conventions now
 >> used in Zope 3 for all new development. The rationale behind it is
 >> that you are forced to document and reason all the cases the
 >> software handles.
 >
 > Most testing I do is not about handling cases at all, it's about
 > testing specific functionality, making sure that old bugs doesn't pop
 >  up again and stuff like that. Most of these tests would in doctest
 > format provide no documentation at all.

Sure, they don't provide /user/ documentation, but it's been my
experience that using doctests for regression and unit tests provides
better /maintainer/ documentation, greatly enhancing their
maintainability.

 > For my calendar, many of the tests are test to do things as migrate
 > data from the old calendar product, making sure that the attendee
 > source for CPS does what it should, but it has no educational use in
 > itself

Again, not for the average user, bit if the hypothetical six month bug
were found and those tests were decently written doctests, it would be
much easier for the maintainer to follow what the test was doing.

 > Many tests require setup/teardown functionality, which gets hard to
 > do in the inherently linear format of doctests.

That's what the setUp and tearDown methods of the test suite are for.

 > Simply put:  I agree doctest is good for testing use cases, and
 > testing documentation. I don't agree that it is any good for testing
 >  anything else.

I (and I think others) will disagree.  Doctest encourages a style of
testing that is superior to old-style unit tests.

 > It  may possible increase the documentation quality marginally,

The intent isn't (necessarily) to increase documentation, but to
encourage tests that are easy to understand and maintain.

 > but it may likewise lower the testing quality.

I don't see why doctest formatted unit, functional, regression, or other
tests would be of lower quality.  It certainly hasn't been my
experience.
-- 
Benji York
Senior Software Engineer
Zope Corporation