[Zope-CMF] Re: Re: CMFTestCase: Best way to create the CMF site?

[Geoff Davis]

On Wed, 05 Oct 2005 12:34:25 +0200, Lennart Regebro wrote:

> Any test including PortalTestCase should really not be seen as a unit
> test, but a fucntional test. ;) If we could put in some effort of
> making a minimal dymmy-portal that can be deleted and recreated very
> quickly, then that would be very interesting. I would assume that that
> involved a lot of work though...

CMFTestCase creates a minimal portal that can be deleted and recreated
relatively quickly.  But it actually is even smarter than that: it uses
the transaction machinery in such a way that it only has to create and
delete the test site once.  See my reply to Chris for an explanation.

The nice thing about CMFTestCase is that it creates an actual CMF site,
not some dummy site whose functionality may or may not be equivalent to
that of a real CMF site.  I think there is a place for using really
stripped-down dummy components.  However, widespread use of dummy
components comes with some real headaches:

* As you note, dummy components take a lot of time to write.
* Dummy components create the need for new tests to ensure that the dummy
components' functionality really does match that of the components they
are replacing.  Do we have such tests in the CMF?  I'm not sure we do.
* Dummy components create the need for additional documentation.  The
absence of such documentation creates barriers to test writing and, as a
result, to the contribution of code to the CMF.

At some point I think we have to trust the stack.  After all, we don't go
around writing dummy versions of python modules such as httplib.  CMFCore
should be able to assume Zope; CMFDefault modules should be able to assume
CMFCore components; products built on CMFDefault should be able to assume
it, etc.

I think the speed issue is a red herring.  I just timed Plone's tests
(almost all of which use PloneTestCase) and CMFCore's tests (all of which
use stripped down dummy components).  The results:

Plone tests: 0.14 sec/test
CMFCore tests: 0.09 sec/test

The dummy components really aren't saving much time.  If you spent the
same amount of work on customer projects that you would spent writing,
documenting, and maintaining a set of good dummy components, I am sure you
could buy a very, very fast computer that would run the tests in no time.

Geoff