How to test changes to ZTK packages?
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions? I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6. BTW, zope.release wants lxml, which is a real pain on Mac OS X and Centos 4. Does the ZTK really need to depend on lxml? Jim -- Jim Fulton Zope Corporation
I have to chime in here too lxml is a real pain and seems to be problematic to get a straightforward build for packages other than ZTK as well. I have had varying success building lxml even under ubuntu - success seems to be dependant on the type of build defined. T On Tue, Jun 30, 2009 at 6:28 PM, Jim Fulton <jim@zope.com> wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
BTW, zope.release wants lxml, which is a real pain on Mac OS X and Centos 4. Does the ZTK really need to depend on lxml?
Jim
-- Jim Fulton Zope Corporation
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Hoffman wrote:
I have to chime in here too
lxml is a real pain and seems to be problematic to get a straightforward build for packages other than ZTK as well. I have had varying success building lxml even under ubuntu - success seems to be dependant on the type of build defined.
While I know that Mac people often report problems building lxml, I never have issues building lxml on Linux: the '--static-deps' option is a sufficient workaround for variants with too-old versions of libxml2 / libxslt. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKSlrV+gerLs4ltQ4RAryqAKCVy3gazZEqD8Vm23B79HCTRGDZFgCaAyfe W8I7h3fmNafq6lQOjV3GZ8c= =QLYw -----END PGP SIGNATURE-----
On Jun 30, 2009, at 2:35 PM, Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Tim Hoffman wrote:
I have to chime in here too
lxml is a real pain and seems to be problematic to get a straightforward build for packages other than ZTK as well. I have had varying success building lxml even under ubuntu - success seems to be dependant on the type of build defined.
While I know that Mac people often report problems building lxml, I never have issues building lxml on Linux: the '--static-deps' option is a sufficient workaround for variants with too-old versions of libxml2 / libxslt.
I'm not familiar with the static-deps option. To what? Its setup.py? It didn't install cleanly for me on Centos 4. (I've generally had difficulty with libxml2 and libxslt on Red Hat based systems,) I'm sure if I screwed around with it, I could get it to work. I don't want to screw around with it just to work on ZTK libraries, which don't actually depend on it. I will if I have to. I ended up using an Ubuntu VM to work on it. I'd prefer not to have to. I'm sad to say this. I think lxml is a worthy project. It's a shame that the system installs of libxml2 and libxslt are such a PITA. I wonder if it would be better for lxml to include it's own copies of these libraries that it built and linked against statically. Will lxml look somewhere in /usr/local? I wonder if hand-built libxml2 and libxslt libraries in /usr/local would make lxml easier to deal with. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
I'm not familiar with the static-deps option. To what? Its setup.py?
$ wget http://pypi.python.org/packages/source/l/lxml/lxml-2.2.2.tar.gz ... $ tar xzf lxml-2.2.2.tar.gz $ cd lxml-2.2.2 $ /path/to/python setup.py bdist_egg --static-deps
It didn't install cleanly for me on Centos 4. (I've generally had difficulty with libxml2 and libxslt on Red Hat based systems,) I'm sure if I screwed around with it, I could get it to work. I don't want to screw around with it just to work on ZTK libraries, which don't actually depend on it. I will if I have to. I ended up using an Ubuntu VM to work on it. I'd prefer not to have to.
I'm sad to say this. I think lxml is a worthy project. It's a shame that the system installs of libxml2 and libxslt are such a PITA. I wonder if it would be better for lxml to include it's own copies of these libraries that it built and linked against statically.
That is what --static-deps' does: it downloads and builds libxml2 / libxslt from source, and links them statically into the .so / .dll / .pyd file for lxml.
Will lxml look somewhere in /usr/local? I wonder if hand-built libxml2 and libxslt libraries in /usr/local would make lxml easier to deal with.
I would just use --static-deps (works well for me on CentOS4, for instance). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKSnrU+gerLs4ltQ4RAi1PAJ9PquHRAFIzvFYpnRayUE9CLDFJvwCeMHKH wr03NK/PQIEX6EmbuvGTPaE= =LB9x -----END PGP SIGNATURE-----
On Jun 30, 2009, at 4:51 PM, Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
I'm not familiar with the static-deps option. To what? Its setup.py?
$ wget http://pypi.python.org/packages/source/l/lxml/lxml-2.2.2.tar.gz ... $ tar xzf lxml-2.2.2.tar.gz $ cd lxml-2.2.2 $ /path/to/python setup.py bdist_egg --static-deps
It didn't install cleanly for me on Centos 4. (I've generally had difficulty with libxml2 and libxslt on Red Hat based systems,) I'm sure if I screwed around with it, I could get it to work. I don't want to screw around with it just to work on ZTK libraries, which don't actually depend on it. I will if I have to. I ended up using an Ubuntu VM to work on it. I'd prefer not to have to.
I'm sad to say this. I think lxml is a worthy project. It's a shame that the system installs of libxml2 and libxslt are such a PITA. I wonder if it would be better for lxml to include it's own copies of these libraries that it built and linked against statically.
That is what --static-deps' does: it downloads and builds libxml2 / libxslt from source, and links them statically into the .so / .dll / .pyd file for lxml.
Ah cool. I could then stuff that in my egg cache. This seems to work for me on Mac OS X. I'm not sure how to run the tests. Unfortunately, the tests aren't included in the egg. Running tests.py in the distro root doesn't work either.
Will lxml look somewhere in /usr/local? I wonder if hand-built libxml2 and libxslt libraries in /usr/local would make lxml easier to deal with.
I would just use --static-deps (works well for me on CentOS4, for instance).
This is a good work around. Thanks! It would be nice to automate this a bit more. Jim -- Jim Fulton Zope Corporation
On Tue, 2009-06-30 at 14:35 -0400, Tres Seaver wrote:
While I know that Mac people often report problems building lxml, I never have issues building lxml on Linux: the '--static-deps' option is a sufficient workaround for variants with too-old versions of libxml2 / libxslt.
I do not know what platform you are running but I have had my share of frustrations with lxml on Fedora and Ubuntu on x86_64. --Tim -- Timothy Cook, MSc Health Informatics Research & Development Services LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook Skype ID == timothy.cook ************************************************************** *You may get my Public GPG key from popular keyservers or * *from this link http://timothywayne.cook.googlepages.com/home* **************************************************************
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Cook wrote:
On Tue, 2009-06-30 at 14:35 -0400, Tres Seaver wrote:
While I know that Mac people often report problems building lxml, I never have issues building lxml on Linux: the '--static-deps' option is a sufficient workaround for variants with too-old versions of libxml2 / libxslt.
I do not know what platform you are running but I have had my share of frustrations with lxml on Fedora and Ubuntu on x86_64.
I haven't had issues with lxml on 32- or 64-bit versions of either Fedora or Ubuntu any time within the last eighteen months or so. I just easy_installed it into a fresh virtualenv on Ubuntu x86_64: $ gunzip -c /usr/share/doc/ubuntu-standard/changelog.gz | head -1 ubuntu-meta (1.124) intrepid; urgency=low $ uname -a Linux mred 2.6.27-14-generic #1 SMP Wed Apr 15 19:29:46 UTC 2009 \ x86_64 GNU/Linux $ /opt/Python-2.6.2/bin/virtualenv --no-site-packages /tmp/lxml ... $ /tmp/lxml/bin/easy_install lxml Searching for lxml Reading http://pypi.python.org/simple/lxml/ Reading http://codespeak.net/lxml Best match: lxml 2.2.2 Downloading http://codespeak.net/lxml/lxml-2.2.2.tgz Processing lxml-2.2.2.tgz Running lxml-2.2.2/setup.py -q bdist_egg --dist-dir \ /tmp/easy_install-Gyfz1B/lxml-2.2.2/egg-dist-tmp-xMuOwQ Building lxml version 2.2.2. NOTE: Trying to build without Cython, pre-generated \ 'src/lxml/lxml.etree.c' needs to be available. Using build configuration of libxslt 1.1.24 Building against libxml2/libxslt in the following directory: /usr/lib Adding lxml 2.2.2 to easy-install.pth file Installed \ /tmp/lxml/lib/python2.6/site-packages/lxml-2.2.2-py2.6-linux-x86_64.egg Processing dependencies for lxml Finished processing dependencies for lxml Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKUOk8+gerLs4ltQ4RAguOAJ4qj5NuKwV3K5K8HMU8lFBFkmBIlQCZAaWO uZ3Zqj0rtXmZpSYp4/cDJ58= =Slt4 -----END PGP SIGNATURE-----
On Tuesday 30 June 2009, Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
Yes, I think testing with zope.release is a good way of doing this right now. I tried to verify the steps: 1. Checkout zope.release 2. Run python bootstrap.py 3. Run ./bin/buildout -N 4. Run ./bin/generate-buildout 5. cd test 6. python ../bootstrap.py 7. ./bin/buildout -N 8. ./bin/test -vpc1 I am running the tests as I am writing this. So far I got one failure: Traceback (most recent call last): File "/opt/zope/packages/eggs/z3c.macro-1.2.1-py2.5.egg/z3c/macro/tests.py", line 29, in <module> import z3c.pt ImportError: No module named pt I'll report the full output when it is done.
BTW, zope.release wants lxml, which is a real pain on Mac OS X and Centos 4. Does the ZTK really need to depend on lxml?
It is needed for the "latest-versions" script as this parses XML. I consider lxml pretty much the standard tool to do XML in Python these days. Who is not using lxml? Having said that, "latest-versions" is not needed by everyone all the time. I could live with putting it into an extra and not build the latest-versions script by default. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Jun 30, 2009, at 1:04 PM, Stephan Richter wrote:
On Tuesday 30 June 2009, Stephan Richter wrote:
I'll report the full output when it is done.
And here it is.
OK, this sorta looks like what I got with Python 2.4. It probably would have been good enough to say "yeah, I get lots of errors too." :) So basically, this *is* how we're supposed to run tests and we're in bad shape, Given that this locks down versions, wouldn't it make more sense to not check in configurations with failing (or hanging) tests? Jim -- Jim Fulton Zope Corporation
On Tuesday 30 June 2009, Jim Fulton wrote:
And here it is.
OK, this sorta looks like what I got with Python 2.4. It probably would have been good enough to say "yeah, I get lots of errors too." :)
Sorry! ;-)
So basically, this is how we're supposed to run tests and we're in bad shape,
Yes.
Given that this locks down versions, wouldn't it make more sense to not check in configurations with failing (or hanging) tests?
That's probably not a bad idea. I think we should make that a rule. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Jun 30, 2009, at 7:25 PM, Stephan Richter wrote:
Given that this locks down versions, wouldn't it make more sense to not check in configurations with failing (or hanging) tests?
That's probably not a bad idea. I think we should make that a rule.
So, I wonder if there is a previous revision of the trunk for which the tests pass for some version of Python. I guess I'll try to discover that. It's a bit mysterious that there are packages for which tests don't even run. Jim -- Jim Fulton Zope Corporation
On 6/30/09 7:03 PM, Stephan Richter wrote:
It is needed for the "latest-versions" script as this parses XML. I consider lxml pretty much the standard tool to do XML in Python these days. Who is not using lxml?
I suspect the majority of people who use OSX as their main platform try to stay as far away from lxml as possible. Not because lxml is bad, but because unless you use special magic it will make your python randomly segfault. This is very sad, and it is not lxml's fault but I see no good solution at this moment. Using z3c.recipe.staticlxml recipe helps a bit for people using buildout, but that is not everyone and even then I have seen random segfaults. Wichert.
Wichert Akkerman wrote:
On 6/30/09 7:03 PM, Stephan Richter wrote:
It is needed for the "latest-versions" script as this parses XML. I consider lxml pretty much the standard tool to do XML in Python these days. Who is not using lxml?
I suspect the majority of people who use OSX as their main platform try to stay as far away from lxml as possible. Not because lxml is bad, but because unless you use special magic it will make your python randomly segfault. This is very sad, and it is not lxml's fault but I see no good solution at this moment.
There is a good solution: binary eggs. lxml already does this for Windows, and we're about to get them for OS X. So I think lxml will again become a reasonable thing to depend on. Which is important, because it's a very good library.
Using z3c.recipe.staticlxml recipe helps a bit for people using buildout, but that is not everyone and even then I have seen random segfaults.
I've never seen errors with a static build (although I've seen the recipe fail to remove a non-static egg in a shared eggs cache). Maybe I'm lucky. :) Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Jun 30, 2009, at 1:03 PM, Stephan Richter wrote:
I consider lxml pretty much the standard tool to do XML in Python these days.
I don't want to diss lxml, but since it's not in the standard library, I imagine far more people use element tree.
Who is not using lxml?
I don't. I don't do much with xml. If I did, I might use it more. I see no reason for a web framework to be all tied up with xml. I wouldn't mind lxml if it wasn't so hard to install.
Having said that, "latest-versions" is not needed by everyone all the time. I could live with putting it into an extra and not build the latest- versions script by default.
+1 Jim -- Jim Fulton Zope Corporation
Can't use lxml on google app engine either. I use ElementTree. If only the lxml extensions to the etree API were available in ElementTree, especially the ability to pretty print the xml and do xpath queries, that would be great. On Tue, Jun 30, 2009 at 7:25 PM, Jim Fulton<jim@zope.com> wrote:
On Jun 30, 2009, at 1:03 PM, Stephan Richter wrote:
I consider lxml pretty much the standard tool to do XML in Python these days.
I don't want to diss lxml, but since it's not in the standard library, I imagine far more people use element tree.
Who is not using lxml?
I don't. I don't do much with xml. If I did, I might use it more. I see no reason for a web framework to be all tied up with xml. I wouldn't mind lxml if it wasn't so hard to install.
Having said that, "latest-versions" is not needed by everyone all the time. I could live with putting it into an extra and not build the latest- versions script by default.
+1
Jim
-- Jim Fulton Zope Corporation
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- Paul Carduner http://www.carduner.net
On Tuesday 30 June 2009, Jim Fulton wrote:
On Jun 30, 2009, at 1:03 PM, Stephan Richter wrote: ...
2. Run python bootstrap.py
Which Python? 2.4, 2.5, 2.6? All of the above?
I have mainly run test under 2.5. I am fine if people run those tests after changes in any Python version. ;-) Of course, if you want to be super-diligent, I think both Python 2.5 and 2.6 should be tested. Regards. Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Jun 30, 2009, at 1:03 PM, Stephan Richter wrote: ...
I am running the tests as I am writing this. So far I got one failure:
Traceback (most recent call last): File "/opt/zope/packages/eggs/z3c.macro-1.2.1-py2.5.egg/z3c/macro/ tests.py", line 29, in <module> import z3c.pt ImportError: No module named pt
I'll report the full output when it is done.
Any news? Or did it hang? (It hangs for me with Pythonj 2.6.) Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
BTW, zope.release wants lxml, which is a real pain on Mac OS X and Centos 4. Does the ZTK really need to depend on lxml?
IMO, Any ZTK package should be testable via plain 'python2.{5,6} setup.py test'. If that doesn't get the testing dependencies installed, then they should be added. There is a recipe for testing *other* packages which might be affected by changes to the package-under-development: http://pypi.python.org/pypi/z3c.recipe.compattest I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKSlxt+gerLs4ltQ4RAgSeAKC41QMLihHrD9JcwmEmXeojL/4kMwCeLEfH ltQDJ4vxrQQH3xBuj8w3uV0= =7xZ4 -----END PGP SIGNATURE-----
* Tres Seaver <tseaver@palladion.com> [2009-06-30 20:41]:
Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
There is a recipe for testing *other* packages which might be affected by changes to the package-under-development: http://pypi.python.org/pypi/z3c.recipe.compattest
This recipe was written at the Grok dependency-cleanup sprint in January with exactly the purpose Jim talks about: testing packages against each other to make sure changes in one don't adversely affect the others. I haven't studied zope.release closely yet, but I think testing-wise it does quite a similar thing, namely running tests for a set of packages. The difference is that compattest uses either pinned version from buildout (but doesn't supply its own list of pins), or alternatively trunk checkouts. Also, it seems to be more lightweight than zope.release both in concept and in usage (mostly since it only does one thing, namely running tests, and not other release management stuff like creating tarballs and uploading them etc.), but I'm biased since I'm one of the compattest authors. For the record, the usage is 1. add it to your buildout, minimally like so: [compattest] recipe = z3c.recipe.compattest 2. run bin/compattest 3. there is no step three. ;-)
I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess).
You can specify include and exclude options; the default is to include a list of zope.* packages that we pulled out of thin air at the sprint, it'd probably be better to use the KGS as a default instead -- but that would duplicate even more zope.release functionality. I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it). My instinct would be to separate KGS handling from tarball creation from testing, that is, remove the test-business from zope.release and use the compattest recipe instead. (Alternatively, retire compattest and have zope.release gain the ability to use trunks so it could be used for continuous integration purposes as well -- but that doesn't feel quite right, to be honest) Thoughts? Wolfgang
On Wed, Jul 1, 2009 at 7:08 AM, Wolfgang Schnerring<ws@gocept.com> wrote:
I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it).
I think it would be good if someone actually would define what the ZTK includes in terms of packages ;) zope.release currently defines some megalomanic set of all zope, z3c and bunch of other things. Some of the z3c packages require lxml which has been seen as a bit tedious to deal with. I don't know when all tests for all packages in zope.release have passed last, but I suspect that has been quite a while for its current trunk. I can still offer http://svn.zope.org/Zope/trunk/versions.cfg?view=markup as a base for a technical definition. That list uses all the latest versions of all packages available on PyPi and I keep it updated for now, as there's no other place where I could get a current ZTK KGS definition from. It includes all packages that are required by Zope2 to both run it and run all of its tests including all tests for all transitive dependencies. Cheers, Hanno
On Jul 1, 2009, at 1:08 AM, Wolfgang Schnerring wrote:
* Tres Seaver <tseaver@palladion.com> [2009-06-30 20:41]:
Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
There is a recipe for testing *other* packages which might be affected by changes to the package-under-development: http://pypi.python.org/pypi/z3c.recipe.compattest
This recipe was written at the Grok dependency-cleanup sprint in January with exactly the purpose Jim talks about: testing packages against each other to make sure changes in one don't adversely affect the others.
I haven't studied zope.release closely yet, but I think testing-wise it does quite a similar thing, namely running tests for a set of packages.
The difference is that compattest uses either pinned version from buildout (but doesn't supply its own list of pins), or alternatively trunk checkouts. Also, it seems to be more lightweight than zope.release both in concept and in usage (mostly since it only does one thing, namely running tests, and not other release management stuff like creating tarballs and uploading them etc.), but I'm biased since I'm one of the compattest authors.
For the record, the usage is 1. add it to your buildout, minimally like so: [compattest] recipe = z3c.recipe.compattest 2. run bin/compattest 3. there is no step three. ;-)
I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess).
You can specify include and exclude options; the default is to include a list of zope.* packages that we pulled out of thin air at the sprint, it'd probably be better to use the KGS as a default instead -- but that would duplicate even more zope.release functionality.
Where is this list? In the recipe?
I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it).
Yes.
My instinct would be to separate KGS handling from tarball creation from testing, that is, remove the test-business from zope.release and use the compattest recipe instead. (Alternatively, retire compattest and have zope.release gain the ability to use trunks so it could be used for continuous integration purposes as well -- but that doesn't feel quite right, to be honest)
Thoughts?
I agree we need one way to do this. I think the KGS concept is right. I think there should be a known good configuration of packages and a way to evolve it. The configuration should be changed only after testing changes. The configuration needs to include Python versions that it is tested with. Beyond that, I don't care what the process is as long as it is documented. Jim -- Jim Fulton Zope Corporation
Hi, On 07/01/2009 11:12 AM, Jim Fulton wrote:
[...]
You can specify include and exclude options; the default is to include a list of zope.* packages that we pulled out of thin air at the sprint, it'd probably be better to use the KGS as a default instead -- but that would duplicate even more zope.release functionality.
Where is this list? In the recipe?
The recipe takes two arguments: - a list of regular expressions to describe packages that should be included - a list of regular expressions to describe packages that should be excluded There is currently a builtin list for each of them that gets appended to the ones you specify when using the recipe in your buildout.
I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it).
Yes.
My instinct would be to separate KGS handling from tarball creation from testing, that is, remove the test-business from zope.release and use the compattest recipe instead. (Alternatively, retire compattest and have zope.release gain the ability to use trunks so it could be used for continuous integration purposes as well -- but that doesn't feel quite right, to be honest)
Thoughts?
I agree we need one way to do this. I think the KGS concept is right. I think there should be a known good configuration of packages and a way to evolve it. The configuration should be changed only after testing changes. The configuration needs to include Python versions that it is tested with. Beyond that, I don't care what the process is as long as it is documented.
The compattest is intended to check the trunks of packages located in one repository like svn.zope.org against the development version of the package you're working on so you can see whether your changes will cause others that depend on you to break. I think it's one of the issues we need to solve in our process and it was actually very valuable when doing the large dependency refactoring at the sprint. If you have a multi-core machine, you can also ask it to run the various test runners in parallel. I was able to execute the 100+ package tests in less than 10 minutes on an 8-core machine. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On Jul 1, 2009, at 1:08 AM, Wolfgang Schnerring wrote:
* Tres Seaver <tseaver@palladion.com> [2009-06-30 20:41]:
Jim Fulton wrote:
I should know this, but I don't. What is the recommended way to test changes to core ZTK packages to mitigate the risk that changes affect other packages? Is there a page somewhere with instructions?
I tried using using zope.release. Building the trunk of zope.release with Python 2.4 and running the tests gives lots of test import errors and test failures. (Lots of tests want to import z3c.pt.) The tests hang when I try to build and run with Python 2.6.
There is a recipe for testing *other* packages which might be affected by changes to the package-under-development: http://pypi.python.org/pypi/z3c.recipe.compattest
This recipe was written at the Grok dependency-cleanup sprint in January with exactly the purpose Jim talks about: testing packages against each other to make sure changes in one don't adversely affect the others.
I haven't studied zope.release closely yet, but I think testing-wise it does quite a similar thing, namely running tests for a set of packages.
The difference is that compattest uses either pinned version from buildout (but doesn't supply its own list of pins), or alternatively trunk checkouts.
So it provides no good configuration to test against. This is a fatal flaw IMO.
Also, it seems to be more lightweight than zope.release both in concept and in usage (mostly since it only does one thing, namely running tests, and not other release management stuff like creating tarballs and uploading them etc.), but I'm biased since I'm one of the compattest authors.
For the record, the usage is 1. add it to your buildout, minimally like so: [compattest] recipe = z3c.recipe.compattest 2. run bin/compattest 3. there is no step three. ;-)
I tried this with the current trunk of zope.app.publisher, which I'm about to modify. I added a compattest part: [compattest] recipe = z3c.recipe.compattest I didn't get bin/compattest, but I did get bin/test_compattest. When I run this, I get lots and lots of errors. :( I could attach them, but what's the point?
I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess).
You can specify include and exclude options; the default is to include a list of zope.* packages that we pulled out of thin air at the sprint, it'd probably be better to use the KGS as a default instead -- but that would duplicate even more zope.release functionality.
I think it would be useful to standardize on *one* compatibility testing method for the ZTK (and then document it).
My instinct would be to separate KGS handling from tarball creation from testing, that is, remove the test-business from zope.release and use the compattest recipe instead. (Alternatively, retire compattest and have zope.release gain the ability to use trunks so it could be used for continuous integration purposes as well -- but that doesn't feel quite right, to be honest)
Thoughts?
I'm kind of stuck at this point. The KGS approach seems most promising because it is based on the idea of a known working configuration. It is still baffling to me that people checked version numbers into the trunk of zope.release that don't pass tests. (Many of the packages there don't even run their tests.) Jim -- Jim Fulton Zope Corporation
On Jun 30, 2009, at 2:41 PM, Tres Seaver wrote: ...
IMO, Any ZTK package should be testable via plain 'python2.{5,6} setup.py test'. If that doesn't get the testing dependencies installed, then they should be added.
There is a recipe for testing *other* packages which might be affected by changes to the package-under-development:
http://pypi.python.org/pypi/z3c.recipe.compattest
I don't know how to supply the set of dependents to that recipe (something in the buildout config file, I guess).
Thanks, I missed this yesterday. Just noticed it when I saw the replies. Jim -- Jim Fulton Zope Corporation
participants (11)
-
Christian Theune -
Hanno Schlichting -
Jim Fulton -
Martin Aspeli -
Paul Carduner -
Stephan Richter -
Tim Cook -
Tim Hoffman -
Tres Seaver -
Wichert Akkerman -
Wolfgang Schnerring