[Zope-dev] Unit testing an application's user interface

[Andrew Wilcox]

Hello everybody, I've made some progress on unit testing my application's
user interface.  I'll share what I've got so far.


For background, there was discussion on unit testing last February on
zope-dev: 

   http://lists.zope.org/pipermail/zope-dev/2000-February/003489.html
   http://lists.zope.org/pipermail/zope-dev/2000-February/003514.html
   http://lists.zope.org/pipermail/zope-dev/2000-February/003501.html

An introduction to why do unit testing can be found at:

   http://www.extremeprogramming.org/rules/unittests.html


First, I do start by splitting out business classes and other functionality
that isn't dependent on Zope, and test these separately in isolation.  This
gives me a solid foundation of working and tested code to build upon.

But I found that I have enough complexity in my user interface that I
really need to test that as well.

I experimented for a time with checking the HTML output of web requests.
The framework would remotely submit http requests, walk through the web
site, exercise various functionality, submit form requests, and check if
the resultant HTML was correct.  Unfortunately, as you might expect, the
tests were quite fragile.  Any change in the website, no matter how small,
would produce different HTML and break all the tests.  I found the HTML
testing of some limited usefulness while refactoring, but I'd have to throw
all the tests away as soon as I added any functionality.

And checking the HTML output isn't good unit testing -- not small, focused
tests of particular functionality.  More thought was needed.

One of the points unit testers make is to focus the time you put into
writing unit tests to target where bugs are most likely.  In my situation,
I found I wasn't being bit by bugs in presentation, in the rendering of
HTML from information in the ZODB.  Where I was getting bugs slipping past
me was in the more complex processing of when my form processing would
update ZODB objects.

With this realization, I came up with an approach that I have found very
useful.  Run a form submit (or other data changing web request) through
Zope, and then check if objects in the ZODB have been updated as you'd expect.

This is nice.  You have a small piece of data going into the test (the form
submit fields), and you can easily check particular attributes of affected
objects in the ZODB.  This is a good unit test: small, predictable, and you
can code the test before implementing the functionality.

It's a good fit for the pattern of bugs that I have in my own code.  I
check the form processing as well as the underlying updating of objects,
because my code tends to have bugs in those areas.  I'm not checking how
the results get rendered (the HTML that comes back), because I usually
don't have bugs in that area when the underlying code is correct and is
getting its own unit testing.  I check if objects in the ZODB are updated
correctly.

As an example, suppose I wanted to unit test the Zope user interface for
editing DTML Documents.  The familiar "Edit" tab has an input box for the
document title and a text box for entering the content.  The form submit is
"manage_edit".  If called remotely, the URL for editing a document named
"TestDocument" would look like
http://www.myzope.com/somepath/TestDocument/manage_edit with the "title"
and "data" fields submitted as form elements.

Here's an external method that will do this test, here checking if the
title is updated properly.

def testChangingDocument(self):
    try:
        document = self.TestDocument
        document.manage_edit(data="Hi, I'm a DTML Document",
                             title="hello",
                             SUBMIT='Change')
        assert document.title == "hello"
    finally:
        get_transaction().abort()
    return "Test was successful"

I call "manage_edit" with the form data, and then check if the ZODB has
been updated as I expect. The transaction abort in the "finally:" clause
ensures that in all cases (whether the test is successful, the assertion
fails, or an error occurs) changes to the ZODB will be backed out of and
the database left unchanged.  This lets me run lots of tests without them
bumping into each other.

My realworld tests tend to be more complicated, first setting up objects
and then checking multiple things, but this is the essential framework.

So far so good.

There are some problems with this method of implementation though.  What I
really want to be doing is setting up the web request with the form fields,
and then running my request through the Zope machinery.  In my example
above, I cheated and looked at the source code for manage_edit() to see how
it wanted to be called.  manage_edit() happens to declare the form fields
it's looking for as arguments to the method, and Zope obligingly pulls
those values out of the request and supplies them.  But manage_edit() could
have been written to pull the values out of the REQUEST.form, and I would
have to call it differently.

Getting the Zope machinery to process the request is trivial, of course, if
I perform an actual web request remotely, such as by using httplib.  But
that makes it hard to check directly if the ZODB was updated in the way I'm
expecting.

So what I need to be doing is to plug into Zope at the right point, say
here's my URL path and request variables, do your thing, get control back,
and then have a chance check out what happened to the affected objects in
the ZODB.

Andrew