[Zope-dev] Circular dependency hell.

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lennart Regebro wrote:
> <This is to a large part an angry rant. Sorry for that.>
> 
> In my effort to port zope.component to Python 3 I constantly encounter
> one problem that is very hard to solve: Circular dependencies. It's
> hard to port module A, when running the tests require module B to be
> ported, and module B depends on module A. Or, as with setuptools, it
> actually uses itself, so it has to work before you can test if it
> works. (Sorry, that's just insane).
> 
> So far the main circularity was that everything depended on
> zope.testing as a testrunner, zc.buildout for making the development
> environment, and zope.testing obviously depended on zope.interface
> etc. I solved that by also adding support for setuptools/distributes
> testrunner and using that instead. That fixed zope.interface,
> zope.event and zope.exception. These modules still have buildout
> configurations if you want them, but you don't need buildout anymore.
> zope.interface, zope.event and zope.exception can now be developed and
> tested with only setuptools, you can run the tests with "python
> setup.py test" both under Python 2 and 3.

Yay!  That is a big win -- I'd like to see us automate testing this way,
so that future development doesn't erode it.  Developing ZTK packages
using buildout should be a convenience, not a mandate.

> But then the time came to zope.testing itself. It, rather annoyingly,
> depends on itself. I did the same there, but almost all tests are even
> more annoyingly, doctests, and depends on a custom version of doctest
> included, etc, etc, etc. The result was hours and hours of minor
> fiddling to find exactly what character in a doctest that made it
> fail. So I've spent an inordinate amount of time on zope.testing, and
> I have come to a conclusion about it:
> 
> == Suggestion 1 ==
> 
> I think we should deprecate zope.testing. Completely. There has been
> some discussion about deprecating it for something else, but I think
> we should just deprecate it. Just say "Don't use zope.testing, it's
> pure evil". We can recommend another testrunner, and it seems nose is
> winning the wars in the Python world there, but I don't think we
> necessarily need to do that. Most testrunners will find the tests by
> themselves, and the tests should be runnable with any testrunner, so
> which one you use is a minor issue.

I'm +0 on this myself.  Certainly I don't think that we should have
packages hard-wire dependencies on zope.testing in order to run their
own tests.

> Instead the big problem with getting rid of zope.testing is that many
> tests use other specific features like the doctest renormalizers etc.
> These could possibly be extracted into separate modules so you can use
> them with other testrunners.

Maybe instead we should strip zope.testing down so that it includes only
these bits (the ones that get imported by third-party test code), and
move the testrunner out to a separate package, e.g. 'zope.testrunner'.
I think Marius Gedminas already suggested such a thing:  we could update
zc.recipe.testrunner to use the new package, which would make existing
buildouts continue to wok.

> But zope.testing is a mess that builds on
> two other messes, namely unittest (yes, it's a bloody mess, the API
> makes no sense) and doctest (which is a horror of a mess). unittest
> and doctest are horrid enough for us to not to make it worse with
> zope.testing in addition.
> 
> The path forward there would simply be to stop depending on
> zope.testing for all ZTK modules. That's probably a bit of work, but I
> think it's worth it.

I believe that the ZTK's charter is to have its packages be widely
reusable, so making them testable without a custom testrunner seems like
 a great idea to me.

> So, with that in mind I today went on to zc.buildout, trying to port
> it to Python 3 by ripping out any usage of zope.testing. Also, the
> standard development mode for zc.buildout is that you run a dev.py
> script, that uses zc.buildout to build a buildout for zc.buildout.
> Obviously, this circularity is the same as I encountered in
> setuptools, and it's evil. But with zc.buildout it's easy to get rid
> off at least as I could use setuptools testrunner instead. But then I
> realize the zc.buildout needs zc.recipe.egg. And zc.recipe.egg depends
> on zc.buildout. Hohum, another circular depedency.

Yup.  I don't even know why zc.recipe.egg is broken out from buildout at
all.

> == Suggestion 2 ==
> 
> Move the egg recipe into zc.buildout itself, and make zc.recipe.egg a
> dummy package that just points at zc.buildout.recipe.egg (or whatever
> we decide to call it). That solves that problem.

+1 from me.  Graphs with cycles are pure evil.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkvIn/oACgkQ+gerLs4ltQ4B0ACfZCum/xeqimodDnJ84AlC9M51
G7kAoJ8p+oBYCkkDOzT7E4VCCfsQmRrq
=ekI6
-----END PGP SIGNATURE-----