Hi there, I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml]. I think they make the graph harder to reason about. That's bad, we want to reason about the system more easily. So I propose that: * we shouldn't create any new "extra" dependencies from now on. * we should investigate ways to remove the need for 'extra' dependencies. The latter point would of course not be done instantly, but if we agree we don't want to create *more* "extra" dependencies we can agree on starting with that policy right away. One place where "extra" dependencies are used quite a lot is in testing. One extra dependency that shows up a lot is a dependency on zope.app.testing. I think that *also* makes things much harder to reason about; testing dependencies should follow normal package dependencies. I therefore think zope.app.testing is one package we should be looking to get rid of eventually by splitting it up among a lot of 'testing' modules in individual packages. This way we won't have zope.app.testing sitting at an edge against our whole dependency tree. Since this is a lot of work this will be an ongoing project but we could at least agree we *want* to do this. Opinions? Regards, Martijn
Martijn Faassen wrote:
I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml].
I think they make the graph harder to reason about. That's bad, we want to reason about the system more easily. So I propose that:
* we shouldn't create any new "extra" dependencies from now on.
* we should investigate ways to remove the need for 'extra' dependencies.
The latter point would of course not be done instantly, but if we agree we don't want to create *more* "extra" dependencies we can agree on starting with that policy right away.
One place where "extra" dependencies are used quite a lot is in testing. One extra dependency that shows up a lot is a dependency on zope.app.testing. I think that *also* makes things much harder to reason about; testing dependencies should follow normal package dependencies.
I therefore think zope.app.testing is one package we should be looking to get rid of eventually by splitting it up among a lot of 'testing' modules in individual packages. This way we won't have zope.app.testing sitting at an edge against our whole dependency tree. Since this is a lot of work this will be an ongoing project but we could at least agree we *want* to do this.
Opinions?
+1 Hanno
2009/3/5 Martijn Faassen <faassen@startifact.com>:
Hi there,
I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml].
I think they make the graph harder to reason about. That's bad, we want to reason about the system more easily. So I propose that:
* we shouldn't create any new "extra" dependencies from now on.
* we should investigate ways to remove the need for 'extra' dependencies.
The latter point would of course not be done instantly, but if we agree we don't want to create *more* "extra" dependencies we can agree on starting with that policy right away.
One place where "extra" dependencies are used quite a lot is in testing. One extra dependency that shows up a lot is a dependency on zope.app.testing. I think that *also* makes things much harder to reason about; testing dependencies should follow normal package dependencies.
I therefore think zope.app.testing is one package we should be looking to get rid of eventually by splitting it up among a lot of 'testing' modules in individual packages. This way we won't have zope.app.testing sitting at an edge against our whole dependency tree. Since this is a lot of work this will be an ongoing project but we could at least agree we *want* to do this.
Opinions?
+99999999 for removing test extras. -0.75 for removing functionality extras. I still don't get how extras are different from additional packages. I'd also like to officially clear things about dependencies for zcml configuration. Most of our packages can be used nicely without any zcml, but the zcml-related dependencies can be quite large. I think that the extras here will do a nice job. I mailed about that a while ago. -- WBR, Dan Korostelev
Dan Korostelev wrote at 2009-3-5 22:14 +0300:
... -0.75 for removing functionality extras. I still don't get how extras are different from additional packages.
I agree with Dan -- and add -1. The extras are equivalent to almost identical additional packages. If this makes reasoning more difficult, expand the graph in the trivial way before you start the reasoning. -- Dieter
On Thu, Mar 5, 2009 at 1:43 PM, Martijn Faassen <faassen@startifact.com> wrote:
* we shouldn't create any new "extra" dependencies from now on.
+1
* we should investigate ways to remove the need for 'extra' dependencies.
+1
I therefore think zope.app.testing is one package we should be looking to get rid of eventually by splitting it up among a lot of 'testing' modules in individual packages. This way we won't have zope.app.testing sitting at an edge against our whole dependency tree.
+1 -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
Previously Martijn Faassen wrote:
I therefore think zope.app.testing is one package we should be looking to get rid of eventually by splitting it up among a lot of 'testing' modules in individual packages. This way we won't have zope.app.testing sitting at an edge against our whole dependency tree. Since this is a lot of work this will be an ongoing project but we could at least agree we *want* to do this.
Opinions?
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
On Thu, Mar 5, 2009 at 5:20 PM, Wichert Akkerman <wichert@wiggy.net> wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
Wichert.
Be aware of nose issue #102: http://code.google.com/p/python-nose/issues/detail?id=102 -- Sidnei da Silva
Previously Sidnei da Silva wrote:
On Thu, Mar 5, 2009 at 5:20 PM, Wichert Akkerman <wichert@wiggy.net> wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
Wichert.
Be aware of nose issue #102:
Is there a particular reason to keep using the test_suite convention? Personally I much prefer nose's habit of automatically picking up tests. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
On Thu, 2009-03-05 at 21:36 +0100, Wichert Akkerman wrote:
Previously Sidnei da Silva wrote:
On Thu, Mar 5, 2009 at 5:20 PM, Wichert Akkerman <wichert@wiggy.net> wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
Wichert.
Be aware of nose issue #102:
Is there a particular reason to keep using the test_suite convention? Personally I much prefer nose's habit of automatically picking up tests.
I think there is not. One of the reasons I refactored the test runner a while ago was to make it possible to support a less-boilerplate-heavy scheme of getting tests set up. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development
Christian Theune wrote:
Wichert. Be aware of nose issue #102:
http://code.google.com/p/python-nose/issues/detail?id=102 Is there a particular reason to keep using the test_suite convention? Personally I much prefer nose's habit of automatically picking up tests.
I think there is not. One of the reasons I refactored the test runner a while ago was to make it possible to support a less-boilerplate-heavy scheme of getting tests set up.
Well okay, but without test_suite, how do you quickly and easilly disable a class full of tests while you work on something else that will break them but that you don't want to deal with just yet? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On 3/23/09 12:57 PM, Chris Withers wrote:
Christian Theune wrote:
Wichert. Be aware of nose issue #102:
http://code.google.com/p/python-nose/issues/detail?id=102 Is there a particular reason to keep using the test_suite convention? Personally I much prefer nose's habit of automatically picking up tests.
I think there is not. One of the reasons I refactored the test runner a while ago was to make it possible to support a less-boilerplate-heavy scheme of getting tests set up.
Well okay, but without test_suite, how do you quickly and easilly disable a class full of tests while you work on something else that will break them but that you don't want to deal with just yet?
nose has a --exclude option Wichert.
On Mon, 2009-03-23 at 14:20 +0100, Wichert Akkerman wrote:
On 3/23/09 12:57 PM, Chris Withers wrote:
Christian Theune wrote:
Wichert. Be aware of nose issue #102:
http://code.google.com/p/python-nose/issues/detail?id=102 Is there a particular reason to keep using the test_suite convention? Personally I much prefer nose's habit of automatically picking up tests.
I think there is not. One of the reasons I refactored the test runner a while ago was to make it possible to support a less-boilerplate-heavy scheme of getting tests set up.
Well okay, but without test_suite, how do you quickly and easilly disable a class full of tests while you work on something else that will break them but that you don't want to deal with just yet?
nose has a --exclude option
I usually just run the tests that I'm interested (-s or -t or a combination) in during those times. I never had to go in and comment out a test_suite function. -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development
Christian Theune wrote at 2009-3-23 14:44 +0100:
... I usually just run the tests that I'm interested (-s or -t or a combination) in during those times. I never had to go in and comment out a test_suite function.
I had when I had run the ZODB test suite. Some tests deterministically had hang -- and I had to exclude them from the suite to get a result from the total suite. -- Dieter
Hi there, Wichert Akkerman wrote: [snip]
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
This is another discussion that has little to do with testing dependencies such as zope.app.testing. I'd therefore like to keep it out of this thread. Regards, Martijn
Wichert Akkerman wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
I'd love to see a side-by-side feature comparison of the major python test discovering and runner frameworks. It would then be interesting to see if there's a compelling enough reason for The Zope Framework steering committee to make a pronouncement on which one to use... cheers, Chris
2009/3/8 Chris Withers <chris@simplistix.co.uk>:
Wichert Akkerman wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
I'd love to see a side-by-side feature comparison of the major python test discovering and runner frameworks. It would then be interesting to see if there's a compelling enough reason for The Zope Framework steering committee to make a pronouncement on which one to use...
+1 -- WBR, Dan Korostelev
Chris Withers wrote:
Wichert Akkerman wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
I'd love to see a side-by-side feature comparison of the major python test discovering and runner frameworks. It would then be interesting to see if there's a compelling enough reason for The Zope Framework steering committee to make a pronouncement on which one to use...
My initial reaction is that the zope.testrunner is enormously featureful, we're using quite a few of those features, and it's being actively maintained. I think going to nose would be a lot of work without compelling new features, and I see little gain in that. I would like setup.py test to be supported somehow if possible. I believe some people have done experiments with this. Tres, do you have any experience with this? Regards, Martijn
On Mar 9, 2009, at 6:32 PM, Martijn Faassen wrote:
Chris Withers wrote:
Wichert Akkerman wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose.
I'd love to see a side-by-side feature comparison of the major python test discovering and runner frameworks. It would then be interesting to see if there's a compelling enough reason for The Zope Framework steering committee to make a pronouncement on which one to use...
My initial reaction is that the zope.testrunner is enormously featureful, we're using quite a few of those features, and it's being actively maintained. I think going to nose would be a lot of work without compelling new features, and I see little gain in that.
I would like setup.py test to be supported somehow if possible.
+1 Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote:
Chris Withers wrote:
Wichert Akkerman wrote:
I would like to see a move away from zope testing frameworks to a more standard testing infrastructure: setup.py test, possibly combined with using nose. I'd love to see a side-by-side feature comparison of the major python test discovering and runner frameworks. It would then be interesting to see if there's a compelling enough reason for The Zope Framework steering committee to make a pronouncement on which one to use...
My initial reaction is that the zope.testrunner is enormously featureful, we're using quite a few of those features, and it's being actively maintained. I think going to nose would be a lot of work without compelling new features, and I see little gain in that.
I would like setup.py test to be supported somehow if possible. I believe some people have done experiments with this. Tres, do you have any experience with this?
I got the ZODB and transaction tests running via the 'eggsupport' module I added to the testrunner:: - ------------------------- 8< ------------------------------- $ svn co $ZSVN/transaction/trunk transaction-trunk A transaction-trunk/LICENSE.txt ... U transaction-trunk Checked out revision 97781. $ cd transaction-trunk/ $ ../bin/python2.6 setup.py test running test Checking .pth file support in . /home/tseaver/projects/Zope-CVS/transaction-trunk/../bin/python2.6 -E -c pass Searching for zope.interface Reading http://pypi.python.org/simple/zope.interface/ Best match: zope.interface 3.5.0 Downloading http://pypi.python.org/packages/source/z/zope.interface/zope.interface-3.5.0... Processing zope.interface-3.5.0.zip Running zope.interface-3.5.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ANScFf/zope.interface-3.5.0/egg-dist-tmp-ut5t2S Installed /home/tseaver/projects/Zope-CVS/transaction-trunk/zope.interface-3.5.0-py2.6-linux-i686.egg Searching for zope.testing Reading http://pypi.python.org/simple/zope.testing/ Reading http://svn.zope.org/zope.testing Best match: zope.testing 3.7.1 Downloading http://pypi.python.org/packages/source/z/zope.testing/zope.testing-3.7.1.tar... Processing zope.testing-3.7.1.tar.gz Running zope.testing-3.7.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-uZFmax/zope.testing-3.7.1/egg-dist-tmp-T243K9 Installed /home/tseaver/projects/Zope-CVS/transaction-trunk/zope.testing-3.7.1-py2.6.egg running egg_info creating transaction.egg-info writing requirements to transaction.egg-info/requires.txt writing transaction.egg-info/PKG-INFO writing top-level names to transaction.egg-info/top_level.txt writing dependency_links to transaction.egg-info/dependency_links.txt writing entry points to transaction.egg-info/entry_points.txt writing manifest file 'transaction.egg-info/SOURCES.txt' writing manifest file 'transaction.egg-info/SOURCES.txt' running build_ext Doctest: transaction.tests.test_SampleResourceManager.ResourceManager ... ok Doctest: transaction.tests.test_SampleResourceManager.ResourceManager.savepoint ... ok Doctest: transaction.tests.test_SampleResourceManager.ResourceManager.tpc_abort ... ok Doctest: transaction.tests.test_SampleResourceManager.ResourceManager.tpc_begin ... ok Doctest: transaction.tests.test_SampleResourceManager.ResourceManager.tpc_finish ... ok Doctest: transaction.tests.test_SampleDataManager.DataManager ... ok Doctest: transaction.tests.test_SampleDataManager.DataManager.abort ... ok Doctest: transaction.tests.test_SampleDataManager.DataManager.commit ... ok Doctest: transaction.tests.test_SampleDataManager.DataManager.prepare ... ok Doctest: transaction.tests.test_SampleDataManager.DataManager.savepoint ... ok Doctest: transaction.tests.test_register_compat ... ok test_as_weakref_list (transaction.tests.test_weakset.WeakSetTests) ... ok test_contains (transaction.tests.test_weakset.WeakSetTests) ... ok test_len (transaction.tests.test_weakset.WeakSetTests) ... ok test_map (transaction.tests.test_weakset.WeakSetTests) ... ok test_remove (transaction.tests.test_weakset.WeakSetTests) ... ok testExceptionInAbort (transaction.tests.test_transaction.TransactionTests) ... No handlers could be found for logger "txn.-1209874240" ok testExceptionInCommit (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcAbort (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcBegin (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcVote (transaction.tests.test_transaction.TransactionTests) ... ok testNSJTransactionAbort (transaction.tests.test_transaction.TransactionTests) ... ok testNSJTransactionCommit (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionAbort (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionCommit (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionNote (transaction.tests.test_transaction.TransactionTests) ... ok Doctest: doom.txt ... ok Doctest: transaction.tests.test_transaction.test_addAfterCommitHook ... ok Doctest: transaction.tests.test_transaction.test_addBeforeCommitHook ... ok Doctest: transaction.tests.test_transaction.test_join ... ok testExceptionInAbort (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInCommit (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcAbort (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcBegin (transaction.tests.test_transaction.TransactionTests) ... ok testExceptionInTpcVote (transaction.tests.test_transaction.TransactionTests) ... ok testNSJTransactionAbort (transaction.tests.test_transaction.TransactionTests) ... ok testNSJTransactionCommit (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionAbort (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionCommit (transaction.tests.test_transaction.TransactionTests) ... ok testTransactionNote (transaction.tests.test_transaction.TransactionTests) ... ok Doctest: savepoint.txt ... ok Doctest: transaction.tests.test_savepoint.testRollbackRollsbackDataManagersThatJoinedLater ... ok - ---------------------------------------------------------------------- Ran 42 tests in 0.152s OK - ------------------------- 8< ------------------------------- Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJtfCw+gerLs4ltQ4RAg3MAKCOvUqhZI35mnaJmtIMjiPDzqwxKACgg5zx ftN/qty034FaqEoacdUShDU= =BHCQ -----END PGP SIGNATURE-----
On Mar 5, 2009, at 1:43 PM, Martijn Faassen wrote:
Hi there,
I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml].
...
Opinions?
I disagree with the blanket statement. I do lean towards not having the extras for the test package only. I'm fine with the policy "If you want zope.testing for your tests, then keep it as a dependency for the package". But I like to have the option of extras for testing additional setups. zc.async uses extras so that the main tests show the functionality as a Python library; another level adds more Zope dependencies, with associated tests; and the last level adds the most. I could have divided these up into multiple teensy-weensy packages but I didn't really want to. It seemed like overkill. I've also done this in other circumstances in which code could use zope.proxy/zope.security, but I really didn't want to add the hard dependency. The tests to show that proxies were handled correctly were only part of the "extras". Dividing this package also would have made no sense--it was already just a few small classes. For a package as central as zope.component, I think the pattern Tres is pursuing--dividing everything up--makes sense. For most other packages, I personally feel that there are circumstances in which extras are a useful tool. I do wonder if there's a ``setup.py test`` spelling for testing these extras though. If there were not, I'd find that a good argument against the "extras" pattern, at least for core Zope packages. Gary
Gary Poster wrote:
On Mar 5, 2009, at 1:43 PM, Martijn Faassen wrote:
Hi there,
I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml].
...
Opinions?
I disagree with the blanket statement.
I do lean towards not having the extras for the test package only. I'm fine with the policy "If you want zope.testing for your tests, then keep it as a dependency for the package".
But I like to have the option of extras for testing additional setups.
zc.async uses extras so that the main tests show the functionality as a Python library; another level adds more Zope dependencies, with associated tests; and the last level adds the most. I could have divided these up into multiple teensy-weensy packages but I didn't really want to. It seemed like overkill.
I've also done this in other circumstances in which code could use zope.proxy/zope.security, but I really didn't want to add the hard dependency. The tests to show that proxies were handled correctly were only part of the "extras". Dividing this package also would have made no sense--it was already just a few small classes.
For a package as central as zope.component, I think the pattern Tres is pursuing--dividing everything up--makes sense.
For most other packages, I personally feel that there are circumstances in which extras are a useful tool.
I do wonder if there's a ``setup.py test`` spelling for testing these extras though. If there were not, I'd find that a good argument against the "extras" pattern, at least for core Zope packages.
It seems there is a 'tests_require' """ If your project's tests need one or more additional packages besides those needed to install it, you can use this option to specify them. It should be a string or list of strings specifying what other distributions need to be present for the package's tests to run. When you run the test command, setuptools will attempt to obtain these (even going so far as to download them using EasyInstall). Note that these required projects will not be installed on the system where the tests are run, but only downloaded to the project's setup directory if they're not already installed locally. """ http://peak.telecommunity.com/DevCenter/setuptools I've seen this used at least here: http://svn.supervisord.org/supervisor/trunk/setup.py Laurence
On Mar 5, 2009, at 5:02 PM, Laurence Rowe wrote:
Gary Poster wrote:
I disagree with the blanket statement.
I do lean towards not having the extras for the test package only. I'm fine with the policy "If you want zope.testing for your tests, then keep it as a dependency for the package".
But I like to have the option of extras for testing additional setups.
zc.async uses extras so that the main tests show the functionality as a Python library; another level adds more Zope dependencies, with associated tests; and the last level adds the most. I could have divided these up into multiple teensy-weensy packages but I didn't really want to. It seemed like overkill. ...
It seems there is a 'tests_require' """ If your project's tests need one or more additional packages besides those needed to install it, you can use this option to specify them. It should be a string or list of strings specifying what other distributions need to be present for the package's tests to run. When you run the test command, setuptools will attempt to obtain these (even going so far as to download them using EasyInstall). Note that these required projects will not be installed on the system where the tests are run, but only downloaded to the project's setup directory if they're not already installed locally. """
Thanks for trying to help, but "tests_require" will not help here. When Gary wrote zc.async, he went to a great length to document it and make it useful for a wide variety of use cases. http://packages.python.org/zc.async/1.5.0/ There are three different zc.async configurations. 1. minimal 2. more 3. everything Each of the use cases above requires the packages for _both_ runtime and tests. The only way to provide all three options to developers is using extras. Which zc.async configuration you will actually use in your application depends on your needs and your application setup. For example, if you list "zc.async [z3]" you'll get the option 3 above. See the docs for zc.async for details, please. Zvezdan
Hey, Laurence Rowe wrote: [snip]
It seems there is a 'tests_require'
One reason that isn't used is that apparently there is no way for us to dig up this information in the way our test runner needs, unlike extras requires. Regards, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote:
Hey,
Laurence Rowe wrote: [snip]
It seems there is a 'tests_require'
One reason that isn't used is that apparently there is no way for us to dig up this information in the way our test runner needs, unlike extras requires.
If the testrunner would require 'eggtestinfo', and introspect the extra data it records, this wouldn't be true any longer. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJsUiO+gerLs4ltQ4RAkrdAJ9BPlqpeVScIMPXAKvc2RjTP1fAiACaAosw 3hd36GBYlzeQFahZzqVA6dM= =UodN -----END PGP SIGNATURE-----
2009/3/6 Tres Seaver <tseaver@palladion.com>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Martijn Faassen wrote:
Hey,
Laurence Rowe wrote: [snip]
It seems there is a 'tests_require'
One reason that isn't used is that apparently there is no way for us to dig up this information in the way our test runner needs, unlike extras requires.
If the testrunner would require 'eggtestinfo', and introspect the extra data it records, this wouldn't be true any longer.
+1. Though, if that "eggtestinfo" package is really tiny, as I believe it is, we could just copy it into the testrunner not to create extra dependencies. -- WBR, Dan Korostelev
On Mar 6, 2009, at 4:30 PM, Dan Korostelev wrote:
2009/3/6 Tres Seaver <tseaver@palladion.com>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Martijn Faassen wrote:
Hey,
Laurence Rowe wrote: [snip]
It seems there is a 'tests_require'
One reason that isn't used is that apparently there is no way for us to dig up this information in the way our test runner needs, unlike extras requires.
If the testrunner would require 'eggtestinfo', and introspect the extra data it records, this wouldn't be true any longer.
+1. Though, if that "eggtestinfo" package is really tiny, as I believe it is, we could just copy it into the testrunner not to create extra dependencies.
(http://pypi.python.org/pypi/eggtestinfo) I skimmed the docs, but they were written generically, so I couldn't confirm: Tres, is this the thing that lets you run ``setup.py test`` against zope.testing, if you configure it properly? The last example seemed like that was it. If so, +1 on getting this into zope.testing, or at least this pattern into our standard ways of writing tests. IME, that's what non-Zope Python developers expect (as Tres has argued before). It's a small thing, but would get the non-Zope Python developer's experience off on the right foot. Tres, you've obviously done the research here to know: can we easily change zope.testing to make that story work? Or would you say that eggtestinfo is still a better/easier way to go about it, at least for now? Gary
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gary Poster wrote:
(http://pypi.python.org/pypi/eggtestinfo)
I skimmed the docs, but they were written generically, so I couldn't confirm: Tres, is this the thing that lets you run ``setup.py test`` against zope.testing, if you configure it properly? The last example seemed like that was it.
At the moment, our testing story can't use things like 'tests_require', etc., because they are only known at the time 'setup()' runs. eggtestinfo addresses this problem by writing introspectable metadata into the "egg info" directory: this information derives from the various testing-related arguments passed to 'setup()'. I worked some on getting ZODB / transaction etc. to run at least most of their tests from 'setup.py test' (those which don't require layer setup). The last example shows the metadata from a package which uses the new 'test_loader' I wrote for this purpose. I also added another setuptools plugin to 'zope.testing', to allow running all tests via 'setup.py ftests'.
If so, +1 on getting this into zope.testing, or at least this pattern into our standard ways of writing tests. IME, that's what non-Zope Python developers expect (as Tres has argued before). It's a small thing, but would get the non-Zope Python developer's experience off on the right foot.
Tres, you've obviously done the research here to know: can we easily change zope.testing to make that story work? Or would you say that eggtestinfo is still a better/easier way to go about it, at least for now?
Somebody likely needs to extend zc.buildout (or a recipe?) to use the information generated by 'eggtestinfo' when building testrunner scripts. My zc.buildout-fu is pretty weak at this point :(. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJsa3N+gerLs4ltQ4RAqHAAKCHtx51GXslMXXBJKVOEC9SkkLTzACgl/IC bPKpvcGR8fwzsZC9mqwjwvA= =8wSw -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dan Korostelev wrote:
2009/3/6 Tres Seaver <tseaver@palladion.com>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Martijn Faassen wrote:
Hey,
Laurence Rowe wrote: [snip]
It seems there is a 'tests_require' One reason that isn't used is that apparently there is no way for us to dig up this information in the way our test runner needs, unlike extras requires. If the testrunner would require 'eggtestinfo', and introspect the extra data it records, this wouldn't be true any longer.
+1. Though, if that "eggtestinfo" package is really tiny, as I believe it is, we could just copy it into the testrunner not to create extra dependencies.
Its job is to register a setuptools plugin which writes extra 'egginfo': it therefore has to be used as a 'setup_requires' dependency (otherwise you have to deal with obscure "first time" failures). I think it would make more sense to leave it around as a tiny dependency which we use everywhere. Making the testrunner smart enough to use the information written by the plugin is probably more than my brain can cope with at 6:00 on a Friday evening. :) Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJsawP+gerLs4ltQ4RAo7CAJ9GZL4vS0BT7D9TtpQmoZ3H+8zvJgCgk6Wo pSFmfNPy87+mexXoSEuvdEM= =sLMs -----END PGP SIGNATURE-----
2009/3/5 Gary Poster <gary.poster@gmail.com>:
On Mar 5, 2009, at 1:43 PM, Martijn Faassen wrote:
Hi there,
I know opinions are divergent about 'extra' dependencies in setup.py. These ar dependencies that effectively make a single project with a single dependency structure into a number of "virtual" packages that each can have a separate list of dependencies. Such a virtual package that we're currently getting rid of is zope.component[zcml].
...
Opinions?
I disagree with the blanket statement.
I do lean towards not having the extras for the test package only. I'm fine with the policy "If you want zope.testing for your tests, then keep it as a dependency for the package".
But I like to have the option of extras for testing additional setups.
zc.async uses extras so that the main tests show the functionality as a Python library; another level adds more Zope dependencies, with associated tests; and the last level adds the most. I could have divided these up into multiple teensy-weensy packages but I didn't really want to. It seemed like overkill.
I've also done this in other circumstances in which code could use zope.proxy/zope.security, but I really didn't want to add the hard dependency. The tests to show that proxies were handled correctly were only part of the "extras". Dividing this package also would have made no sense--it was already just a few small classes.
For a package as central as zope.component, I think the pattern Tres is pursuing--dividing everything up--makes sense.
For most other packages, I personally feel that there are circumstances in which extras are a useful tool.
A strong +1 on that explanation. -- WBR, Dan Korostelev
Hi there, Thanks Gary for sketching our the zc.async usecase. Note that zc.async isn't in the Zope Framework at this point in time so it wouldn't be directly affected by this policy, but it's still a useful usecase of course. We seem to have a split community here... I care about this as it is more easy to see through dependency graphs and reason about circular dependencies when we don't have extra dependencies floating around, and because extras sometimes allow us to hide complicated dependency relationships (such as with zope.app.testing). Since cleaning out dependencies is such an important issue in the Framework I'd like to see less of them... So what about the following proposal then? The intent is to handle this on a case by case basis so we *shrink* the amount of extra dependencies in use within the framework. * we are going to work at getting rid of the zope.app.testing extra by distributing its facilities into individual zope.* packages. Hopefully we can get a clear consensus on this one from the people who object to the general policy. * if you think a new "extra" dependency is needed in a Zope Framework package, and you're not just moving stuff between packages but actually developing new code, bring it up on zope-dev so we can at least consider alternatives. Perhaps a better home can be found for this code. * on a case-by-case basis we can consider removing extra dependencies for particular Zope Framework packages, just like what we're doing right now for zope.component. We'll discuss that whenever needed. I'll of course be biased in favor of such removal, but I can be convinced otherwise. How do people feel about this? Note again it only applies to Framework packages, so shouldn't affect everybody. Regards, Martijn
On Mar 6, 2009, at 9:50 AM, Martijn Faassen wrote:
Hi there,
Thanks Gary for sketching our the zc.async usecase. Note that zc.async isn't in the Zope Framework at this point in time so it wouldn't be directly affected by this policy, but it's still a useful usecase of course.
Right, that was the intent. ...
* we are going to work at getting rid of the zope.app.testing extra by distributing its facilities into individual zope.* packages. Hopefully we can get a clear consensus on this one from the people who object to the general policy.
+1
* if you think a new "extra" dependency is needed in a Zope Framework package, and you're not just moving stuff between packages but actually developing new code, bring it up on zope-dev so we can at least consider alternatives. Perhaps a better home can be found for this code.
Fair enough.
* on a case-by-case basis we can consider removing extra dependencies for particular Zope Framework packages, just like what we're doing right now for zope.component. We'll discuss that whenever needed. I'll of course be biased in favor of such removal, but I can be convinced otherwise.
Fair enough. Thanks Gary
participants (15)
-
Benji York -
Chris Withers -
Christian Theune -
Dan Korostelev -
Dieter Maurer -
Fred Drake -
Gary Poster -
Hanno Schlichting -
Jim Fulton -
Laurence Rowe -
Martijn Faassen -
Sidnei da Silva -
Tres Seaver -
Wichert Akkerman -
Zvezdan Petkovic