[Zope-dev] Functional testing enhancement suggestion

[Lennart Regebro]

I'm doing quite a bit of functional testing now, and although it's good
for making sure there are no errors, I find it slightly lacking in the
two otehr uses of tests: Debugging, and test driven development.

The reason is simple: When a page publishing fails, it's hard to figure
out WHY. What I'd want is a traceback of the error. Of course, these are
gonna be big and ugly, being publishing tracebacks and all, but it's
better than just getting to know that "503 != 200", something that I'm
pretty well aware of. ;-)

So, I tried to find if I could get the exception, or traceback or
exc_info, or something similar from the returned RESPONSE object, but I
failed. I do get an HTML-formatted traceback in the Body, but that's not 
very useful as it's completely unreadable.


If isn't there, could we add it? Currently I do

     from zExceptions.ExceptionFormatter import format_exception
     t, v, tb = sys.exc_info()
     tb = format_exception(t, v, tb, as_html=0)
     response._text_traceback = '\n'.join(tb)

in the exception handling of Test.py's publish_module(). That works
fine, and only affects testing, which seems like a good idea. Of course, 
it could also be added in HTTPResponse.exception(), but that method is 
horrendously big and complex already.

I also added the following method to my testcase:

     def assertResponse(self, response, status):
         if not response.getStatus() == status:
             tb =  getattr(response._response, '_exc_info', None)
             if tb:
                 raise self.failureException, tb

and now, when I run the functional tests, instead of doing a

     "self.failUnlessEqual(response.getStatus(), 200)"

and getting the error message "AssertionError, 503 != 200" I do

     "self.asserResponse(response, 200)"

and get a traceback of what actually caused the error! Very handy!

Thoughts? Opinions? The suggested changes above are small, only affects 
testing (as it's in Test.py and functional.py only) and is completely 
backwards compatible. Zope 2.8 even, maybe? ;-)

-- 
Lennart Regebro, Nuxeo     http://www.nuxeo.com/
CPS Content Management     http://www.cps-project.org/