Jim Fulton wrote at 2008-3-11 09:58 -0400:
On Mar 11, 2008, at 9:46 AM, Andreas Jung wrote: ...
You can also document your edgecase in the same way as comments within your unittests.
This is true in theory, but in practice, it almost never happens.
There is no guarantee that doctests will contain useful text, but they encourage text which has a greater chance of being useful than no text at all.
"DocTests" essentially provide examples. I find documentation that only consists of examples very bad documentation -- not concise, at the wrong abstraction level, expecting me to induce the concepts from the examples rather than to describe the concepts and *then* (maybe) make some examples. Faced with the Zope 3/inspect incompatibility ("https://bugs.launchpad.net/zope3/+bug/181371") I have hit such a bad documentation: I have two classes "C1" and "C2", "C2" inherits from "C1". "C1" magically gets an attribute "a", but "C2" does not. Moreover, "C2.a" raises an AttributeError. There is a DocTest, that shows me that "C2.a" should indeed raise an "AttributeError". Thus, I can see that the author had something in mind when he implemented this counter intuitive behaviour. But, there is nothing that tells me what this may have been and why "C1" is treated differently from "C2". Documentation and tests have different purposes: Documentation tells relevant facts to the reader. It should be on an adequate abstraction level. It should clearly describe the concepts and include examples at places where this is helpful to unterstand the concepts. It should not consist of examples only and leave the concepts undescribed. The primary purpose of tests it to ensure that there are few undetected bugs. Some tests need to go to the same detail level as the implementation itself in order to probe for bugs introduced by implementation edgecases -- a level too detailed for a good documentation (apart from source code documentation itself). If a good test coveradge it obtained by performing the same tests over and over again on different data sets, representing relevant uses cases, the better. I do not need and I do not want essentially the same documentation for each of these sets. The ZODB test suite makes very good use of this principle, executing e.g. the "ConflictResolution" tests on both "FileStorage" and "ZEOStorage". The documentation is concise and clear: class FileStorageTests(..., ConflictResolutionTests, ...): ... By the way, a good documentation uses the same principle. It simply states "a 'FileStorage' supports conflict resolution (see XXXX for a description of the 'conflict resolution concept)" and does not show examples for conflict resolution for each an every conflict resolving storage. -- Dieter