[Zope-dev] Re: zope.sendmail Retry fixes and new state machine.

[Dieter Maurer]

Jim Fulton wrote at 2008-3-11 09:58 -0400:
>
>On Mar 11, 2008, at 9:46 AM, Andreas Jung wrote:
>...
>> You can also document your edgecase in the same way as comments  
>> within your
>> unittests.
>
>This is true in theory, but in practice, it almost never happens.
>
>There is no guarantee that doctests will contain useful text, but they  
>encourage text which has a greater chance of being useful than no text  
>at all.

"DocTests" essentially provide examples.
I find documentation that only consists of examples very bad
documentation -- not concise, at the wrong abstraction level,
expecting me to induce the concepts from the examples rather
than to describe the concepts and *then* (maybe) make some
examples.


Faced with the Zope 3/inspect incompatibility
("https://bugs.launchpad.net/zope3/+bug/181371")
I have hit such a bad documentation:

  I have two classes "C1" and "C2", "C2" inherits from "C1".
  "C1" magically gets an attribute "a", but "C2" does not.
  Moreover, "C2.a" raises an AttributeError.

  There is a DocTest, that shows me that "C2.a" should indeed
  raise an "AttributeError". Thus, I can see that the author
  had something in mind when he implemented this counter intuitive
  behaviour.
  But, there is nothing that tells me what this may have been
  and why "C1" is treated differently from "C2".


Documentation and tests have different purposes:

 Documentation tells relevant facts to the reader. It should be
 on an adequate abstraction level. It should clearly
 describe the concepts and include examples at places where
 this is helpful to unterstand the concepts.
 It should not consist of examples only and leave the concepts
 undescribed.

 The primary purpose of tests it to ensure that there are few
 undetected bugs. Some tests need to go to the same detail level
 as the implementation itself in order to probe for bugs
 introduced by implementation edgecases -- a level too detailed
 for a good documentation (apart from source code documentation itself).

 If a good test coveradge it obtained by performing the same
 tests over and over again on different data sets, representing
 relevant uses cases, the better. I do not need and I do not
 want essentially the same documentation for each of these sets.
 The ZODB test suite makes very good use of this principle,
 executing e.g. the "ConflictResolution" tests on
 both "FileStorage" and "ZEOStorage".
 The documentation is concise and clear:
 class FileStorageTests(..., ConflictResolutionTests, ...): ...

 By the way, a good documentation uses the same principle.
 It simply states "a 'FileStorage' supports conflict resolution (see
 XXXX for a description of the 'conflict resolution concept)"
 and does not show examples for conflict resolution for each an
 every conflict resolving storage.



-- 
Dieter