-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
On Sep 3, 2008, at 8:37 PM, Tres Seaver wrote:
The change under discussion about testing is whether or not to remove the test requirements as part of 'install_requires' in setup.py. Some folks have objecctions to using the setuptools-provided 'tests_require' field, although I think the argument there is weak: packages which spell 'tests_require' and 'test_suite' in their setup can be tested trivially with 'python setup.py test', which seems a win to me.
I agree that this is a win. It is not a big win IMO, as I prefer a separate test runner.
It would be far more useful to me if the data were captured as part of egg info.
I'm not volunteering to write it, but it strikes me as odd that folks haven't morphed zc.buildout (and / or associated recipes) to use this field. I *did* write a setuptools add on which saved the 'tests_require' info into the EGG_INFO directory: that package could be used to capture the metadata during installation of packages, for consumption by a testrunner later.
That's very good. It would be much much much better if setuptools would do this by default. Now that setuptools' development is opening up, perhaps we can do this.
'egginfo' uses the established "setuptools plugin" API: I don't have commit access, but I would be fine with having it merged. In the meantime, including it within 'zc.buildout' environments would make the information widely available.
The setuptools add-on doesn't help when you don't have source releases and many interesting packages get distributed as eggs on Windows.
'egginfo' includes the test info in bdist_egg zipfiles if it is present on the system where the egg is built.
I discovered today I think the time is ripe for a "blank buffer" rewrite of the testrunner: it is so full of "twisty passages" that my confidence in its own internal correctness is seriously depleted. It has too many features,
I doubt it. I find the vast majority of it's features very useful. I agree that the test discovery is ripe for a revamp.
Can you truly say that reading the code doesn't make your eyes cross? Here are a couple I noted while debugging broken discovery yesterday: - Extensive use of generators, in spite of the fact that we always consume them all. It feels like a "here's a new language feature, let's use it" choice, and makes debugging the code harder. I doubt very much that the decision was driven by an actual constraint (the only one I can imagine is RAM consumption). - '--path' vs '--test-path' vs. '--package-path' interact in unpredictable ways, based on ordering.
and too many incompatible ways to run it: I would love a simpler version, whose discovery logic used egg metadata instead of (mal)heuristics (e.g., ordering of '--test-path' and '--package-path' arguments can make some tests unfindable).
I'm very open to revamping test discovery.
I'm opposed to a rewrite. I sure as heck wouldn't be willing to spend time on a rewrite. I wouldn't be opposed to replacing out test runner with a bunch of nose plugins that added missing features. I have almost zero interest in using setuptools test runner during development.
I typically use 'setup.py test' in two cases: - When evaluating a new package, I create a virtualenv and run 'setup.py develop' and 'setup.py test' using the virtualenv's python. Packages whose tests run clealy in a sandbox get a higher "score" than those which don't: for one thing, it guarantees that their dependencies are accurate. - When building packages which I hope will be used outsize of Zopeland, I want to make the tests accessible to folks who don't use zc.buildout or zope.testing. I tend to skip running 'setup.py test' only in projects where I am writing custom software, and where the project is already committed to another testrunner. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIv/4u+gerLs4ltQ4RAp+8AJ9iTagUjGOdkgYGIA7xGqeyFu6X/wCcD74d 6LZrrCg9OsDb8+FYHG2rVmM= =d2kX -----END PGP SIGNATURE-----