[Zope-CMF] Re: CachingPolicyManager improvements

[Raphael Ritz]

yuppie wrote:
> Florent Guillaume wrote:
[..]
>> However this kind of default fixture actually is quite heavy, and 
>> makes you write tests that are hardly "unit" anymore. They are still 
>> good integration tests, but I've found that using PortalTestCase or 
>> something like that is often extremely costly in terms of time spent 
>> running the tests.
> 
> 
> Agreed. Nevertheless we should focus on lowering the barrier of entry 
> for new CMF contributors. People used to write tests for Zope or Plone 
> will have less trouble if they can use ZopeTestCase. And ZopeTestCase 
> tests are definitely better than no tests at all.
> 
> On the other hand I'm not sure if it is a good idea to use 
> PortalTestCase (shipped with ZopeTestCase):
> 
> PortalTestCase is in the wrong layer. It makes assumptions about the way 
> CMF works. Changes to the CMF might break PortalTestCase and create a 
> dependency on a new Zope release. (We saw that problem already with the 
> _refreshSkinData changes in CMF 1.5)

Even if this might be considered off topic if not plain wrong:
For me this raises the more fundamental issue of unit tests
versus integration tests versus functional tests.
IMHO there are many cases in which we (ab)use the
unit testing framework to do what's conceptually at least an
integration test if not a functional test.

I acknowledge that it is not easy to draw the boarder line here but
in general a unit test in the strict sense shouldn't depend (much)
on the framework whereas an integration test of course has to.

Considering in particular PloneTestCase as a basis for integration
tests therefore explains its "heavy" nature but also points out that
it's not needed (nor deemed appropriate) for core unit tests.

Maybe there should even be a discussion of "what to move up"?
By that I mean: now that we have a more and more accepted and
integrated functional testing framework
(Selenium/Zelenium/PloneSelenium/...)
some of the more complex "unit" tests today might actually be
turned into full-blown functional tests (and maybe some more
light-weight, real unit test).

And as Yuppie pointed out: we should try hard to make
it as easy as possible to provide tests in the first place.
This could and should include some detailed best-practice
examples together with some concise explanation of the
basic concepts. While I consider it (conceptually) easy
to write plain Python unit tests or full-blown, (Plone)Selenium
based functional tests I keep on having a hard time figuring
out what and how to test when writing add-on products for
CMF/Plone. I usually start with testing the product installation
in a CMF/Plone site which by it's very nature is of course not
a unit but an integration test.

Just my 2 cents.

Raphael

PS: This is about the level at which I look at this:
http://www.bccn-berlin.de/People/ritz/sd2005/testing



> 
> Cheers, Yuppie
> 
> 
> _______________________________________________
> Zope-CMF maillist  -  Zope-CMF at lists.zope.org
> http://mail.zope.org/mailman/listinfo/zope-cmf
> 
> See http://collector.zope.org/CMF for bug reports and feature requests
>