[Zope-dev] Why I dislike narrative doctests

[Sidnei da Silva]

Can't you start a plain-old unittests module and stick your test in
there? Is there any rule that says you cannot create a plain-old test
module if the project you are fixing the bug in uses doctests?

On Wed, Apr 23, 2008 at 5:19 PM, Marius Gedminas <mgedmin at b4net.lt> wrote:
> Suppose I find a bug (say, zope.testing.testrunner.StartUpFailure
>  objects make unittest.TextTestRunner cry).  Being a good developer I
>  want to start the bug fix with a unit test.
>
>  Now if zope.testing used old-style isolated unit tests, I could open the
>  tests.py (or tests/test_testrunner.py) in a text editor and start
>  writing:
>
>     class OptionsStub(object):
>         post_mortem = False
>
>     def doctest_StartUpFailure():
>         """Test that StartUpFailure is a proper unittest.TestCase
>
>         StartUpFailure is a TestCase-lookalike that is inserted into the
>         test suite to indicate that a failed import of a test module
>         won't be unnoticed.
>
>             >>> from zope.testing.testrunner import StartUpFailure
>             >>> options = OptionsStub()
>             >>> module_name = 'test_me'
>             >>> exc_info = (ImportError, ImportError('zonkulator'), None)
>             >>> test = StartUpFailure(options, module_name, exc_info)
>             >>> unittest.TextTestRunner(verbosity=2).run(test)
>
>         This is a regression test for http://launchpad.net/bugs/221151
>         """
>
>  There, I'm almost done.  Now I can fix the bug and paste the correct
>  output into the doctest.
>
>  But zope.testing uses narrative doctests instead of isolated unit tests.
>  There are 22 plain-text files that together comprise over 4500 lines of
>  text.  Now instead of mechanically opening the appropriate test module
>  and grepping for the class/function name I have to figure out which of
>  the txt files is the appropriate place for the new test.  After that I
>  have to figure out how to insert it into the narrative seamlessly.  Or
>  decide to create a new file---but a hundred of .txt files each 20-lines
>  long is not a good idea, in my opinion.
>
>  Suddenly I almost don't want to fix the bug any more.
>
>  While I was writing this, I suddenly realised that almost all
>  zope.testing's txt files are actually functional tests.  There are
>  nearly no unit tests there.  Perhaps that's what causing me pain.
>
>  I still maintain that:
>
>   * adding a new unit test should be a trivial task
>
>   * extending a narrative doctest is not a trivial task
>
>   * the only way to solve this conflict is to explicitly separate unit
>     tests (test_*.py) from doctest-enhanced API documentation (*.txt).
>
>  Marius Gedminas
>  --
>  UNIX is user friendly. It's just selective about who its friends are.
>
> -----BEGIN PGP SIGNATURE-----
>  Version: GnuPG v1.4.6 (GNU/Linux)
>
>  iD8DBQFID5m0kVdEXeem148RAqs5AJ4qUo9IiqMSsuFOIrZBTq+xyWv/FQCePM/Q
>  eTvxFfGfDPuGA1YzlM8CTks=
>  =J91E
>  -----END PGP SIGNATURE-----
>
> _______________________________________________
>  Zope-Dev maillist  -  Zope-Dev at zope.org
>  http://mail.zope.org/mailman/listinfo/zope-dev
>  **  No cross posts or HTML encoding!  **
>  (Related lists -
>   http://mail.zope.org/mailman/listinfo/zope-announce
>   http://mail.zope.org/mailman/listinfo/zope )
>
>



-- 
Sidnei da Silva
Enfold Systems                http://enfoldsystems.com
Fax +1 832 201 8856     Office +1 713 942 2377 Ext 214