[Zope-Coders] Unit testing issues, redux

[Tres Seaver]

I am trying to puzzle out the reason for the two "No test suite
found in file" problems (usually, that is a sign that a module
scope import blew beets before 'def test_suite' could be executed).

One interesting piece of data:

   $ cvs -q up -AdP
   ...
   $ python2.1 wo_pcgi.py
   ...
   $ python2.1 utilities/testrunner.py -qa
   (same result?)
   $ python2.1 utilities/testrunner.py -d lib/python/Products
   testOverlappingSids (testSessionDataManager.TestMultiThread) ... Exception in 
thread Thread-47:
   Traceback (most recent call last):
     File "/usr/local/python2.1/lib/python2.1/threading.py", line 378, in 
__bootstrap
       self.run()
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/Sessions/tests/testSessionDataManager.py", 
line 339, in run
       self.run1()
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/Sessions/tests/testSessionDataManager.py", 
line 367, in run1
       data = session_data_manager.getSessionData()
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/Sessions/SessionDataManager.py", 
line 155, in getSessionData
       return self._getSessionDataObject(key)
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/Sessions/SessionDataManager.py", 
line 238, in _getSessionDataObject
       ob = container.new_or_existing(key)
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/Transience/Transience.py", 
line 244, in new_or_existing
       if not item.isValid():
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/TemporaryFolder/LowConflictConnection.py", 
line 106, in setstate
       p, serial = self._storage.load(oid, self._version)
     File 
"/home/tseaver/projects/Zope-head/lib/python/Products/TemporaryFolder/TemporaryStorage.py", 
line 155, in load
       s=self._index[oid]
   KeyError: ;

   ok


Rerunning causes 'testOverlappingSids' to pass  Running a
thrird time recreates the problem, except with 'Thread-22'
the culprit (ick, race conditions are hell!).  These tests
take a *long* time to run -- are they really unit tests?

Also, when run this way, three of the PageTemplate tests bomb
due to permission problems.  Methinks somebody is calling
'newSecurityManager' somewhere in a test without calling
'noSecurityManager' in teardown.

   $ find . -name "test*.py" | xargs grep "n[eo][w]*SecurityManager"

points the finger at 'lib/python/Products/PythonScripts/tests/testPythonScript.py'.

More hacking shows that PageTemplates have a subtle dependency
on whether 'Zope' has ever been imported;  they use a dummy
security manager if not, which means that their tests run fine
in isolation, but not following any test which may have imported
Zope.  I have updated them to set a permissive security policy
(and restore it in tearDown).

We probably need to add some knobs to testrunner to allow sanity
checking critical global state (presence of a security manager,
etc.) betweeen tests (not to be run every time, but when such
issues arise).

Update:  it turns out that 'lib/python/OFS/tests/testCopySupport'
was setting a security policy and not clearing it.  I am checking
in a fix for that now.


Tres.
-- 
===============================================================
Tres Seaver                                tseaver@zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com