[Zope3-dev] The Zope Software Certification Program and Common Repository Proposal

[Lennart Regebro]

Thanks for the answer. I only have one remaining comment, then, about testing:

On 2/21/06, Stephan Richter <srichter at cosmos.phy.tufts.edu> wrote:
> Why is it random? It is taken straight from the conventions now used in Zope 3
> for all new development. The rationale behind it is that you are forced to
> document and reason all the cases the software handles.

Most testing I do is not about handling cases at all, it's about
testing specific functionality, making sure that old bugs doesn't pop
up again and stuff like that. Most of these tests would in doctest
format provide no documentation at all. For my calendar, many of the
tests are test to do things as migrate data from the old calendar
product, making sure that the attendee source for CPS does what it
should, but it has no educational use in itself, the API for is is
tested in CalCore and CalZope and that is indeed in DocTest format (at
least partly), we also have tests to check that the translations are
consistent, and stuff like that. Many tests require setup/teardown
functionality, which gets hard to do in the inherently linear format
of doctests. For example, the upgrade tests need to create old
calendars at the setup. Each test then fills the calendars with
different types of data, and migrates the calendar. This migration
includes installation of the new software and replacing of a local
utility, which then needs to be undone before the next test. I would
eitehr need to duplicate this code many times, or test all cases in
one fat migration step, which would make it much harder to figure out
exactly what failed.

Simply put:  I agree doctest is good for testing use cases, and
testing documentation. I don't agree that it is any good for testing
anything else.

It is indeed good practice to start your development with doing
usecases, expanding them to be doctest and using this to drive
dvelopment. But once the code has matured past that stage, I see no
reason to require that all tests should be doctests, or you should
need to convince people why doctests are impossible. It  may possible
increase the documentation quality marginally, but it may likewise
lower the testing quality.

--
Lennart Regebro, Nuxeo     http://www.nuxeo.com/
CPS Content Management     http://www.cps-project.org/