[Zope-dev] Circular dependency hell.

[Martin Aspeli]

Hi Lennart & co,

On 17 April 2010 02:38, Lennart Regebro <regebro at gmail.com> wrote:
> On Fri, Apr 16, 2010 at 19:53, Jonathan Lange <jml at mumak.net> wrote:
>> As the author of one of those other testrunners, I can tell you that
>> if you do this you'll find that your number one biggest problem is
>> getting layers to work.
>
> Ah, right, layers are in zope.testing too. To actually get widespread
> movement from zope.testing we would have to make some other layer
> support. Hm...

As you may know, I've been working on plone.testing. This is mainly to
make it easier for people working on Plone packages to write "good"
tests (and a lot of it is just about patterns, rather than code), but
in fact it doesn't depend on Plone (and only soft-depends on Zope 2).
I'm certainly hoping to use it for my "plain Zope 3/Toolkit" packages
in the future.

plone.testing makes very heavy use of layers. I think layers are a
great feature, when done properly, and I haven't seen any better
approach. The setUpClass/setUpModule stuff in unittest2 is nice, but
doesn't really solve the same problem. For integration testing with
something as complex as Zope 2 (or, arguably, the ZODB, various bits
of the ZTK, and so on) layers allow "us" the framework authors to make
life much easier for those people who are not experts in the
framework.

Anyway, a few high level observations:

 - In neither plone.testing (apart from its own tests), nor in code
that uses it, would I imagine actually depending on zope.testing via
an import. We use unittest(2) and doctest from the standard library.

 - We do depend on a zope.testing-compatible test runner, in that we
expect layers to work. In reality, we depend on zc.recipe.testrunner,
though I would *love* to be able to do 'python setup.py test' as well
(and have that execute tests with layers). I have no idea how that
works or whether it'd be possible.

 - The way zope.testing promotes writing layers is actually pretty
evil. It overloads the 'class' keyword, essentially, making you spell
"base layers" as base classes. This has a few problems:

   - If your "base layer" has a testTearDown method, say, and your
layer doesn't, the base class version will actually inherit into the
child layer. zope.testing will then run the same code twice, once for
the base layer and once for the child layer.

   - You can't use OOP inheritance to re-use layer conveniences.

   - People get quite confused. :)

   - You can't have two copies of a layer - all layers are
module-level global singletons. This becomes somewhat important when
layers manage and expose resources (like database connections).

Anyway, based on Ross Patterson's work in collective.testcaselayer, I
think we have something better in plone.testing's layer module.

General info and patterns:
http://svn.plone.org/svn/plone/plone.testing/trunk/README.txt
Layer module: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.py
Layer doctests:
http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.txt

If I could have my cake and eat it too, I'd like:

 - A test runner that supports layers
 - A test runner that supports the new fixture methods in
unittest2/Python 2.7 - setUpClass and setUpModule
 - A test runner that properly reports skipped tests (pretty easy)
 - A test runner that supports the above begin run from 'setup.py test'
 - A test runner installable via a buildout part just like zc.recipe.testrunner

We could quite possibly factor layer.py out of plone.testing and push
it into something else if that was desirable. It's self-contained and
has no dependencies.

Martin