* Tres Seaver <tseaver@palladion.com> [2009-06-30 20:41]:
Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
There is a recipe for testing *other* packages which might be affected by changes to the package-under-development: http://pypi.python.org/pypi/z3c.recipe.compattest
This recipe was written at the Grok dependency-cleanup sprint in January with exactly the purpose Jim talks about: testing packages against each other to make sure changes in one don't adversely affect the others. I haven't studied zope.release closely yet, but I think testing-wise it does quite a similar thing, namely running tests for a set of packages. The difference is that compattest uses either pinned version from buildout (but doesn't supply its own list of pins), or alternatively trunk checkouts. Also, it seems to be more lightweight than zope.release both in concept and in usage (mostly since it only does one thing, namely running tests, and not other release management stuff like creating tarballs and uploading them etc.), but I'm biased since I'm one of the compattest authors. For the record, the usage is 1. add it to your buildout, minimally like so: [compattest] recipe = z3c.recipe.compattest 2. run bin/compattest 3. there is no step three. ;-)
I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess).
You can specify include and exclude options; the default is to include a list of zope.* packages that we pulled out of thin air at the sprint, it'd probably be better to use the KGS as a default instead -- but that would duplicate even more zope.release functionality. I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it). My instinct would be to separate KGS handling from tarball creation from testing, that is, remove the test-business from zope.release and use the compattest recipe instead. (Alternatively, retire compattest and have zope.release gain the ability to use trunks so it could be used for continuous integration purposes as well -- but that doesn't feel quite right, to be honest) Thoughts? Wolfgang