[Zope-CMF] unit test weirdness.

[Mark McEahern]

[Chris Withers]
> That's because neither of those is the canonical way of running
> CMF unit tests.
>
> They should be run using:
>
> python all_cmf_tests.py [-q]
>
> ...from your Products directory.

Hmm, all_cmf_tests.py does a plain import of Zope, assuming it is already
started.  That works, I guess, if you're running ZEO.  I'm not.  So here's
what I'm seeing (appended to the end below).

Q:  Is this a bug in CMF?  I think it is, but perhaps I don't understand the
design philosophy behind Zope unit tests.  I've always assumed each unit
test should be isolated, capable of being run by itself without a lot of
rigamarole.  The notion that there's a "canonical" way to run the tests
(forcing you to run them in a big suite) seems contrary to that.  I would
think the canonical way to run any unit test would be:

  python testname.py

where testname.py is a single module with one or more TestCase instances in
it.  That's not contrary to having a way to run them all at once--but your
statement about canonical suggests a design philosophy that makes a false
dichotomy between running one test and running them all.

Of course, what's likely is that there's something I just don't understand.
Please help me.

Thanks,

// m

***
Naive attempt to run canonical test driver:
***

$ sudo python2.1 all_cmf_tests.py
Traceback (most recent call last):
  File "all_cmf_tests.py", line 74, in ?
    main()
  File "all_cmf_tests.py", line 70, in main
    unittest.main(defaultTest='test_suite')
  File "/usr/local/lib/python2.1/unittest.py", line 663, in __init__
    self.parseArgs(argv)
  File "/usr/local/lib/python2.1/unittest.py", line 690, in parseArgs
    self.createTests()
  File "/usr/local/lib/python2.1/unittest.py", line 696, in createTests
    self.module)
  File "/usr/local/lib/python2.1/unittest.py", line 458, in
loadTestsFromNames
    suites.append(self.loadTestsFromName(name, module))
  File "/usr/local/lib/python2.1/unittest.py", line 443, in
loadTestsFromName
    test = obj()
  File "all_cmf_tests.py", line 18, in test_suite
    from Products.CMFCore.tests.base.utils import build_test_suite
  File "/usr/local/zope/lib/python/Products/CMFCore/__init__.py", line 18,
in ?
    import PortalObject, PortalContent, PortalFolder
  File "/usr/local/zope/lib/python/Products/CMFCore/PortalObject.py", line
18, in ?
    from Globals import InitializeClass
  File "/var/tmp/src/Zope-2.6.1-src/lib/python/Globals.py", line 23, in ?
    import Acquisition, ComputedAttribute, App.PersistentExtra, os
  File "/var/tmp/src/Zope-2.6.1-src/lib/python/App/PersistentExtra.py", line
15, in ?
    from Persistence import Persistent
ImportError: cannot import name Persistent

***
Edit all_cmf_tests.py to do Zope.startup() after import Zope
***

$ diff all_cmf_tests.py all_cmf_tests_2.py
17c17
<     import Zope
---
>     import Zope; Zope.startup()

***
Run modified test driver...
***

$ sudo python2.1 all_cmf_tests_2.py
............................................................................
..........................F.................................................
............................................................................
.........................
======================================================================
FAIL: test_index_html (Products.CMFCore.tests.test_FSImage.FSImageTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_FSImage.py", line 64, in test_index_html
  File "/usr/local/lib/python2.1/unittest.py", line 273, in failUnlessEqual
    raise self.failureException, (msg or '%s != %s' % (first, second))
AssertionError: 209 != 209
----------------------------------------------------------------------
Ran 253 tests in 25.700s

FAILED (failures=1)

-