Is there an equivalent for "tests_require" in buildout.cfg/ zc.recipe.testrunner? The reason I ask is this... currently the zope.18n distribution has an install_requires that looks something like this: install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component [zcml]', 'zope.configuration', ] That's actually a no-good pack of lies, though. ;-) I'd like to change setup.py to something like this: test_suite = "__main__.alltests", # to support "setup.py test" tests_require=[ 'zope.testing', 'pytz', 'zope.component [zcml]', 'zope.configuration', ], install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component', # *we* do not require zope.configuration at all. Our tests # do (for silly reasons) and the framework which uses our zcml # does. ], But I want buildout tests to continue to work. I suspect the rest of this discussion is going to hurt. ;-) - C
On Wednesday 14 November 2007, Chris McDonough wrote:
That's actually a no-good pack of lies, though. ;-) I'd like to change setup.py to something like this:
test_suite = "__main__.alltests", # to support "setup.py test" tests_require=[ 'zope.testing', 'pytz', 'zope.component [zcml]', 'zope.configuration', ], install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component', # *we* do not require zope.configuration at all. Our tests # do (for silly reasons) and the framework which uses our zcml # does. ],
But I want buildout tests to continue to work. I suspect the rest of this discussion is going to hurt. ;-)
Right now we are using extra_requires that has the "test" key, which we specify for the test runner. We have discussed using tests_require before, but this would need some new software in buildout. I would *love* to see this implemented. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Chris McDonough wrote:
Is there an equivalent for "tests_require" in buildout.cfg/zc.recipe.testrunner?
The reason I ask is this... currently the zope.18n distribution has an install_requires that looks something like this:
install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component [zcml]', 'zope.configuration', ]
That's actually a no-good pack of lies, though. ;-) I'd like to change setup.py to something like this:
test_suite = "__main__.alltests", # to support "setup.py test" tests_require=[ 'zope.testing', 'pytz', 'zope.component [zcml]', 'zope.configuration', ], install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component', # *we* do not require zope.configuration at all. Our tests # do (for silly reasons) and the framework which uses our zcml # does. ],
But I want buildout tests to continue to work.
-1 for using different methods for running tests. Testing packages should be done in same way by all developers. If one person run tests in one way and another one in another way is not good. IMO, we should have a policy to use Buildout for development of Zope packages. Regards, Baiju M
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Baiju M wrote:
Chris McDonough wrote:
Is there an equivalent for "tests_require" in buildout.cfg/zc.recipe.testrunner?
The reason I ask is this... currently the zope.18n distribution has an install_requires that looks something like this:
install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component [zcml]', 'zope.configuration', ]
That's actually a no-good pack of lies, though. ;-) I'd like to change setup.py to something like this:
test_suite = "__main__.alltests", # to support "setup.py test" tests_require=[ 'zope.testing', 'pytz', 'zope.component [zcml]', 'zope.configuration', ], install_requires=[ 'setuptools', 'pytz', 'zope.i18nmessageid', 'zope.component', # *we* do not require zope.configuration at all. Our tests # do (for silly reasons) and the framework which uses our zcml # does. ],
But I want buildout tests to continue to work.
-1 for using different methods for running tests.
Testing packages should be done in same way by all developers. If one person run tests in one way and another one in another way is not good.
IMO, we should have a policy to use Buildout for development of Zope packages.
Chris is not advocating switching away from zc.buildout for the *development* of the packages; he is arguing that we can't force people who *use* the packages to stop using the facilities supplied setuptools. If we want people outside the rather small group of core Zope developers to *use* our software, we are going to need to accomodate their expectations about how Python pacakges work. At a miniumum, any package which we release to the Cheeseshop should be testable and installable using the "standard" tool used by nearly everyone: setuptools. That means we need to work harder at a coule of things: - Making sure the install-time dependencies in setup.py are accurate, minimal, and themselves installable. $ wget http://download.zope.org/zope.tal-3.4.0.tar.gz $ tar xzf zope.tal-3.4.0.tar.gz $ cd zope.tal-3.4.0 $ /path/to/python setup.py install If that fails for *any* package listed on the cheeseshop, then we lose: people won't use it, and they won't bother looking at other packages we release, either. - Making sure that the tests can psss for somebody who is not a core Zope developer, (and therefore who doesn't know about zc.buildout, or want to). E.g.: $ /path/to/python setup.py develop $ /path/to/python setup.py test If there are dependencies which are needed to run the tests but not merely to install the package, there should be a way to spell that. In setuptools, that is what 'tests_require' is for; buildout doesn't (yet) have such a spelling (I think). In order to succeed here, we are going to have to quit releasing packages without more careful thought, testing, etc. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPFkm+gerLs4ltQ4RAtLpAJsEag5KlwI/upMn2lh8uSCneAT08QCdFk2/ VqlhGdbHT0jV9EMPjoaOZ+w= =3g3v -----END PGP SIGNATURE-----
On Nov 15, 2007, at 9:35 AM, Tres Seaver wrote: ...
Chris is not advocating switching away from zc.buildout for the *development* of the packages; he is arguing that we can't force people who *use* the packages to stop using the facilities supplied setuptools.
Agreed,
If we want people outside the rather small group of core Zope developers to *use* our software, we are going to need to accomodate their expectations about how Python pacakges work. At a miniumum, any package which we release to the Cheeseshop should be testable and installable using the "standard" tool used by nearly everyone: setuptools.
That means we need to work harder at a coule of things:
- Making sure the install-time dependencies in setup.py are accurate, minimal, and themselves installable.
Yup
- Making sure that the tests can psss for somebody who is not a core Zope developer, (and therefore who doesn't know about zc.buildout, or want to). E.g.:
$ /path/to/python setup.py develop $ /path/to/python setup.py test
Grrrrr. I find "setup.py test" to be really annoying as it requires meta data that is available only to that command.
If there are dependencies which are needed to run the tests but not merely to install the package, there should be a way to spell that.
Agreed. I'll note that there is a school of thought that I proposed but am somewhat ambivalent about that anything required by tests should also be required by install. This is in accordance with "fly what you test, test what you fly." To the degree that we do this, then there is no need for tests_require.
In setuptools, that is what 'tests_require' is for; buildout doesn't (yet) have such a spelling (I think).
Ideally, buildout (really the testrunner recipe) should be able to honor tests_require, but setuptools doesn't make this information available to any other packages. :( (big frowny) Also, I believe that more is required to get "setup.py test" to work. I'm unsure what the details are. When we do this, we tend to use a test extra. I suggest that when we use a test extra, we also put the value in tests_require. I will find this so distasteful that it will help me stick to the strategy of just putting these dependencies in install_requires. :)
In order to succeed here, we are going to have to quit releasing packages without more careful thought, testing, etc.
+1 This should not be interpreted as criticism of the heroic and well- meaning efforts to date. Despite that inevitable mistakes that were made, I'm very appreciative of the efforts a few individual people made to make the transition to eggs possible. I also apologize for not providing more oversight. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
On Nov 15, 2007, at 9:35 AM, Tres Seaver wrote: ...
Chris is not advocating switching away from zc.buildout for the *development* of the packages; he is arguing that we can't force people who *use* the packages to stop using the facilities supplied setuptools.
Agreed,
If we want people outside the rather small group of core Zope developers to *use* our software, we are going to need to accomodate their expectations about how Python pacakges work. At a miniumum, any package which we release to the Cheeseshop should be testable and installable using the "standard" tool used by nearly everyone: setuptools.
That means we need to work harder at a coule of things:
- Making sure the install-time dependencies in setup.py are accurate, minimal, and themselves installable.
Yup
- Making sure that the tests can psss for somebody who is not a core Zope developer, (and therefore who doesn't know about zc.buildout, or want to). E.g.:
$ /path/to/python setup.py develop $ /path/to/python setup.py test
Grrrrr. I find "setup.py test" to be really annoying as it requires meta data that is available only to that command.
Not after running 'egg_info', or any other setuptools-aware command; the tests_require dependencies then become introspectable. E.g.: $ svn co $ZSVN/zope.tal/trunk zope.tal-trunk $ cd zope.tal-trunk $ /path/to/setuptools-aware/python setup.py egg_info $ cat src/zope.tal-egg-info/requires.txt setuptools zope.i18nmessageid zope.interface [test] zope.testing
If there are dependencies which are needed to run the tests but not merely to install the package, there should be a way to spell that.
Agreed. I'll note that there is a school of thought that I proposed but am somewhat ambivalent about that anything required by tests should also be required by install. This is in accordance with "fly what you test, test what you fly." To the degree that we do this, then there is no need for tests_require.
There are a bunch of packages out there which contain ZCML, and try to test the loadability of it. Those tests end up creating a huge furball of dependencies which are artificial: the package itself doesn't need to process ZCML, and any application which wants to conume the ZCML provided by the package is going to have those dependencies satisfied anyway.
In setuptools, that is what 'tests_require' is for; buildout doesn't (yet) have such a spelling (I think).
Ideally, buildout (really the testrunner recipe) should be able to honor tests_require, but setuptools doesn't make this information available to any other packages. :( (big frowny) Also, I believe that more is required to get "setup.py test" to work. I'm unsure what the details are.
See above. If buildout can trigger creation of the egg-info directory, all should be well. In fact, that directory will already be present in any source distribution built with setuptools.
When we do this, we tend to use a test extra. I suggest that when we use a test extra, we also put the value in tests_require. I will find this so distasteful that it will help me stick to the strategy of just putting these dependencies in install_requires. :)
In order to succeed here, we are going to have to quit releasing packages without more careful thought, testing, etc.
+1
This should not be interpreted as criticism of the heroic and well- meaning efforts to date. Despite that inevitable mistakes that were made, I'm very appreciative of the efforts a few individual people made to make the transition to eggs possible. I also apologize for not providing more oversight.
Agreed completely (on the heroic part). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPGtF+gerLs4ltQ4RAiflAKCjtH9ZutX/rL3v4w3qC6antxWUMQCfWgmU AFNKLCpInKK9xNB3U2Aliws= =ooWa -----END PGP SIGNATURE-----
On Nov 15, 2007, at 10:52 AM, Tres Seaver wrote: ...
Grrrrr. I find "setup.py test" to be really annoying as it requires meta data that is available only to that command.
Not after running 'egg_info', or any other setuptools-aware command; the tests_require dependencies then become introspectable. E.g.:
$ svn co $ZSVN/zope.tal/trunk zope.tal-trunk $ cd zope.tal-trunk $ /path/to/setuptools-aware/python setup.py egg_info $ cat src/zope.tal-egg-info/requires.txt setuptools zope.i18nmessageid zope.interface
[test] zope.testing
None of these is tests_require. The above lists the contents of install_requires and the test extra. zope.tal is odd because it lists the same values in tests_require and install requires. Try putting something else in tests_require and see if you see it show up. I know that the contents of tests_require isn't saved because I *tried* to get buildout to use it and asked about it on the distutils sig: http://mail.python.org/pipermail/distutils-sig/2007-January/ 007083.html Of course, something could have changed since then, but I doubt it. ...
Ideally, buildout (really the testrunner recipe) should be able to honor tests_require, but setuptools doesn't make this information available to any other packages. :( (big frowny) Also, I believe that more is required to get "setup.py test" to work. I'm unsure what the details are.
See above. If buildout can trigger creation of the egg-info directory, all should be well. In fact, that directory will already be present in any source distribution built with setuptools.
Again, I believe you are mistaken. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
On Nov 15, 2007, at 10:52 AM, Tres Seaver wrote: ...
Grrrrr. I find "setup.py test" to be really annoying as it requires meta data that is available only to that command. Not after running 'egg_info', or any other setuptools-aware command; the tests_require dependencies then become introspectable. E.g.:
$ svn co $ZSVN/zope.tal/trunk zope.tal-trunk $ cd zope.tal-trunk $ /path/to/setuptools-aware/python setup.py egg_info $ cat src/zope.tal-egg-info/requires.txt setuptools zope.i18nmessageid zope.interface
[test] zope.testing
None of these is tests_require. The above lists the contents of install_requires and the test extra. zope.tal is odd because it lists the same values in tests_require and install requires. Try putting something else in tests_require and see if you see it show up.
I know that the contents of tests_require isn't saved because I *tried* to get buildout to use it and asked about it on the distutils sig:
http://mail.python.org/pipermail/distutils-sig/2007-January/007083.html
Of course, something could have changed since then, but I doubt it.
Nope.
Ideally, buildout (really the testrunner recipe) should be able to honor tests_require, but setuptools doesn't make this information available to any other packages. :( (big frowny) Also, I believe that more is required to get "setup.py test" to work. I'm unsure what the details are. See above. If buildout can trigger creation of the egg-info directory, all should be well. In fact, that directory will already be present in any source distribution built with setuptools.
Again, I believe you are mistaken.
You are correct: I was misled by the presence of the section for the 'tests' extra. Attached find a patch against the setuptools-0.6 branch which addes a 'tests_require.txt' file entry point. I will propose that the patch be included over on the distutils list. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPHR++gerLs4ltQ4RAnVTAKC7KSP2UTNjQt8aCVwDeBW80Zu0UgCfT+/T DBdr4PuHBOBmBHP4Gte2R8U= =2M92 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
You are correct: I was misled by the presence of the section for the 'tests' extra. Attached find a patch against the setuptools-0.6 branch which addes a 'tests_require.txt' file entry point. I will propose that the patch be included over on the distutils list.
I realized (and Phillip Eby reminded me) that setuptools plugins can be distributed separately: I just released 'eggtestinfo' to the cheeseshop: when the plugin is installed, generated '.egg-info' directories will contain a 'test_info.txt' file, which documents both the 'test_suite' and the 'tests_require' from 'setup.py'. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHPH8G+gerLs4ltQ4RAgzTAJsFMY3I+TAkb6ZEN8x28Dow9VidLQCgzVM7 tBQKIULZo88KrA9l1wGldgw= =So7f -----END PGP SIGNATURE-----
Jim Fulton wrote at 2007-11-15 10:28 -0500:
... Agreed. I'll note that there is a school of thought that I proposed but am somewhat ambivalent about that anything required by tests should also be required by install. This is in accordance with "fly what you test, test what you fly." To the degree that we do this, then there is no need for tests_require.
"AdvancedQuery" is a counter example to that. "AdvancedQuery" is willing to be used with almost all indexes. Thus, "install_requires" should not require any special indexes. But, "AdvancedQuery" can better work with "Managable Indexes" and, then, provides more features. Therefore, its test suite uses "Managable Index" and examines the "more freatures". -- Dieter
On Nov 15, 2007, at 12:44 PM, Dieter Maurer wrote:
Jim Fulton wrote at 2007-11-15 10:28 -0500:
... Agreed. I'll note that there is a school of thought that I proposed but am somewhat ambivalent about that anything required by tests should also be required by install. This is in accordance with "fly what you test, test what you fly." To the degree that we do this, then there is no need for tests_require.
"AdvancedQuery" is a counter example to that.
"AdvancedQuery" is willing to be used with almost all indexes. Thus, "install_requires" should not require any special indexes.
But, "AdvancedQuery" can better work with "Managable Indexes" and, then, provides more features. Therefore, its test suite uses "Managable Index" and examines the "more freatures".
ALso, often the set of dependencies required by the tests are larger than the set of dependencies required for the distribution to be useful at installation time. E.g. in one of my distributions, I ship a set of ZCML declaration definitions, and my tests need to test it, and thus the tests need some ZCML package support to run. But the distribution doesn't actually itself need the same ZCML support packages to run, it just needs it to test that, were someone to use the ZCML declaration stuff I've created, it would work. - C
On Nov 15, 2007 9:35 AM, Tres Seaver <tseaver@palladion.com> wrote:
- Making sure that the tests can psss for somebody who is not a core Zope developer, (and therefore who doesn't know about zc.buildout, or want to). E.g.:
$ /path/to/python setup.py develop $ /path/to/python setup.py test
How many people actually use "setup.py test" for a package they aren't developing? Generally, when I download a package, I don't expect to know a priori how to run the tests. Given the number of testing frameworks and tools commonly used for Python these days (nose, trial, py.test, etc.), I'm certainly not going to assume that the setuptools solution was chosen. Given that both setuptools and "bare" distutils use setup.py as the interface to build & install operations, I'm not going to assume that a package uses setuptools. I think of setuptools as a tool for a package's developer to use in setup.py, which may color my thinking in odd ways. Generally, though, I don't know setuptools is being used unless I actually look inside setup.py. I'm not suggesting that "setup.py test" is a bad thing, just that I wouldn't assume it. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
Previously Fred Drake wrote:
On Nov 15, 2007 9:35 AM, Tres Seaver <tseaver@palladion.com> wrote:
- Making sure that the tests can psss for somebody who is not a core Zope developer, (and therefore who doesn't know about zc.buildout, or want to). E.g.:
$ /path/to/python setup.py develop $ /path/to/python setup.py test
How many people actually use "setup.py test" for a package they aren't developing?
Generally, when I download a package, I don't expect to know a priori how to run the tests. Given the number of testing frameworks and tools commonly used for Python these days (nose, trial, py.test, etc.), I'm certainly not going to assume that the setuptools solution was chosen.
nose hooks perfectly in setup.py test. In fact I generally run nose that way. It should be possible to hook the zope testrunner into the setup.py test_suite hook as well. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
participants (8)
-
Baiju M -
Chris McDonough -
Dieter Maurer -
Fred Drake -
Jim Fulton -
Stephan Richter -
Tres Seaver -
Wichert Akkerman