unit test policy questions
Hi! Traditionally the last two lines of unit test files look like this: if __name__ == '__main__': unittest.main(defaultTest='test_suite') That makes it easy to run the tests of a specific file. But it doesn't work with tests that require the zope testrunner. AFAICS something like this is needed instead: if __name__ == '__main__': from zope.testing.testrunner import run run(['-m', 'test_foo', '--test-path', '.']) Questions: ---------- 1.) Is it still policy to add these lines? 2.) Is there a better solution for using zope testrunner than the one shown above? Cheers, Yuppie
On 29 July 2010 19:26, yuppie <y.2010@wcm-solutions.de> wrote:
Hi!
Traditionally the last two lines of unit test files look like this:
if __name__ == '__main__': unittest.main(defaultTest='test_suite')
That makes it easy to run the tests of a specific file. But it doesn't work with tests that require the zope testrunner. AFAICS something like this is needed instead:
if __name__ == '__main__': from zope.testing.testrunner import run run(['-m', 'test_foo', '--test-path', '.'])
Questions: ----------
1.) Is it still policy to add these lines?
2.) Is there a better solution for using zope testrunner than the one shown above?
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically. Martin
Martin Aspeli wrote:
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? Cheers, Yuppie
Am 29.07.2010, 14:14 Uhr, schrieb yuppie <y.2010@wcm-solutions.de>:
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
I try and limit my tests to the package or module I'm working on. I've been bitten by my occasional use and the changes in some of the switches but -s gives you most of the control you need. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
On 29 July 2010 20:14, yuppie <y.2010@wcm-solutions.de> wrote:
Martin Aspeli wrote:
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
See the -s and -t options. :) Martin
On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli <optilude+lists@gmail.com> wrote:
On 29 July 2010 20:14, yuppie <y.2010@wcm-solutions.de> wrote:
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
See the -s and -t options. :)
Don't forget -m which runs exactly one module (file). I always use commands like: bin/test -s zope.package bin/test -s zope.package -m test_module bin/test -s zope.package -m test_module -t test_some_method Especially since unittest and zope.testing can even discover TestSuites inside modules, it seems any kind of explicit configuration in the test files isn't needed anymore. Hanno
Hi! Hanno Schlichting wrote:
On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli<optilude+lists@gmail.com> wrote:
On 29 July 2010 20:14, yuppie<y.2010@wcm-solutions.de> wrote:
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
See the -s and -t options. :)
Don't forget -m which runs exactly one module (file).
I always use commands like:
bin/test -s zope.package bin/test -s zope.package -m test_module bin/test -s zope.package -m test_module -t test_some_method
If everybody is using this, nobody is using CMF.buildout for development. Anyway. I fixed CMF.buildout: http://svn.zope.org/?rev=115204&view=rev
Especially since unittest and zope.testing can even discover TestSuites inside modules, it seems any kind of explicit configuration in the test files isn't needed anymore.
Do you think it should become policy to remove "def test_suite():" where possible? I'm not sure if I like that because it is less explicit. Cheers, Yuppie
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 7/29/10 15:57 , yuppie wrote:
Hanno Schlichting wrote:
On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli<optilude+lists@gmail.com> wrote:
On 29 July 2010 20:14, yuppie<y.2010@wcm-solutions.de> wrote:
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? See the -s and -t options. :) Don't forget -m which runs exactly one module (file).
I always use commands like:
bin/test -s zope.package bin/test -s zope.package -m test_module bin/test -s zope.package -m test_module -t test_some_method
If everybody is using this, nobody is using CMF.buildout for development.
Huh? I use it. I just don't use "-m" myself. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxRibYACgkQRAx5nvEhZLIE7wCfePls1EWIlJ5JlXmkk36kw1+y ZJIAoLpNA7ukrhN/bOV5ZfiCf+GhUbXY =h567 -----END PGP SIGNATURE-----
On Thu, Jul 29, 2010 at 3:57 PM, yuppie <y.2010@wcm-solutions.de> wrote:
If everybody is using this, nobody is using CMF.buildout for development. Anyway. I fixed CMF.buildout: http://svn.zope.org/?rev=115204&view=rev
I'm using CMF.buildout and was always annoyed by it overriding the -m option. But the CMF tests are fast enough, so I just ran the tests for a package at a time.
Do you think it should become policy to remove "def test_suite():" where possible? I'm not sure if I like that because it is less explicit.
I wouldn't make any decision at this point. Once we actually use unittest2 inside zope.testing we might want to reevaluate this. Hanno
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 yuppie wrote:
Hi!
Hanno Schlichting wrote:
On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli<optilude+lists@gmail.com> wrote:
On 29 July 2010 20:14, yuppie<y.2010@wcm-solutions.de> wrote:
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? See the -s and -t options. :) Don't forget -m which runs exactly one module (file).
I always use commands like:
bin/test -s zope.package bin/test -s zope.package -m test_module bin/test -s zope.package -m test_module -t test_some_method
If everybody is using this, nobody is using CMF.buildout for development. Anyway. I fixed CMF.buildout: http://svn.zope.org/?rev=115204&view=rev
+1 for removing the 'if __name__ == "__main__"' boilerplate opportunistically.
Especially since unittest and zope.testing can even discover TestSuites inside modules, it seems any kind of explicit configuration in the test files isn't needed anymore.
Do you think it should become policy to remove "def test_suite():" where possible? I'm not sure if I like that because it is less explicit.
I don't believe that zope.testing's testrunner works without 'def test_suite()'. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkxRkb0ACgkQ+gerLs4ltQ4KugCgtZa1XK2SRcMIeu4e0BlZZOQ4 t2gAnjPZjjPewRWr0OnxOeezDn0crgb5 =8VTG -----END PGP SIGNATURE-----
On 29 July 2010 22:35, Tres Seaver <tseaver@palladion.com> wrote:
I don't believe that zope.testing's testrunner works without 'def test_suite()'.
Latter versions can detect unittest.TestCase-derived test suites automatically. For doctests you still need test_suite(). Martin
On Thu, Jul 29, 2010 at 4:35 PM, Tres Seaver <tseaver@palladion.com> wrote:
I don't believe that zope.testing's testrunner works without 'def test_suite()'.
Oh it does. Ever since 3.8.0 from mid 2009: - Testrunner automatically picks up descendants of unittest.TestCase in test modules, so you don't have to provide a test_suite() anymore. This just doesn't work for doctests and various forms of auto generating test cases and other fancier constructs. Hanno
On 7/29/10 16:35 , Tres Seaver wrote:
yuppie wrote:
Do you think it should become policy to remove "def test_suite():" where possible? I'm not sure if I like that because it is less explicit.
I don't believe that zope.testing's testrunner works without 'def test_suite()'.
It seems to pick up my tests without it. Wichert.
Hi Martin! Martin Aspeli wrote:
On 29 July 2010 20:14, yuppie<y.2010@wcm-solutions.de> wrote:
Martin Aspeli wrote:
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
See the -s and -t options. :)
Well. It's not that simple. Today I tried to modify Products.GenericSetup.tests.test_tool using CMF.buildout/trunk. I usually use the -m option, but that doesn't work in CMF.buildout because it already uses that option. So I tried to execute the module and noticed that's currently broken. The -s option doesn't allow to run a specific module. And if I use the -t option (-ttest_tool), most tests fail :( Cheers, Yuppie
On Thu, Jul 29, 2010 at 02:52:36PM +0200, yuppie wrote:
Hi Martin!
Martin Aspeli wrote:
On 29 July 2010 20:14, yuppie<y.2010@wcm-solutions.de> wrote:
Martin Aspeli wrote:
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files?
See the -s and -t options. :)
Well. It's not that simple. Today I tried to modify Products.GenericSetup.tests.test_tool using CMF.buildout/trunk.
I usually use the -m option, but that doesn't work in CMF.buildout because it already uses that option.
Ouch. Not a good idea, in my book.
So I tried to execute the module and noticed that's currently broken.
The -s option doesn't allow to run a specific module.
And if I use the -t option (-ttest_tool), most tests fail :(
That is an indication that the tests aren't independent. I would consider such a test suite to be broken. Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
Hi! Marius Gedminas wrote:
And if I use the -t option (-ttest_tool), most tests fail :(
That is an indication that the tests aren't independent. I would consider such a test suite to be broken.
Well. In this case the tests are independent from each other, but not from the way tests are filtered: The -m filter is applied *before* test modules are imported. The -t filter is applied *after* importing all available test modules. Some nasty code in Zope 2 behaves differently if different modules are imported before running the tests. Cheers, Yuppie
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 7/29/10 14:00 , Martin Aspeli wrote:
On 29 July 2010 19:26, yuppie <y.2010@wcm-solutions.de> wrote:
1.) Is it still policy to add these lines?
2.) Is there a better solution for using zope testrunner than the one shown above?
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Same here. I have never in my life run tests by executing the module. The options provided by the testrunner to pare down what I want to test have always been sufficient for me. I have had a tendency to add these blurbs myself, thinking "I guess it's useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, it's not really useful anymore. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxRcUYACgkQRAx5nvEhZLLubgCgpCyXpHVToolxYzsfGGubmdHi n+YAnjCVZvY0WhoWZ09015unDWvnMIAx =O8yt -----END PGP SIGNATURE-----
On 07/29/2010 02:17 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 7/29/10 14:00 , Martin Aspeli wrote:
On 29 July 2010 19:26, yuppie<y.2010@wcm-solutions.de> wrote:
1.) Is it still policy to add these lines?
2.) Is there a better solution for using zope testrunner than the one shown above?
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Same here. I have never in my life run tests by executing the module. The options provided by the testrunner to pare down what I want to test have always been sufficient for me.
I have had a tendency to add these blurbs myself, thinking "I guess it's useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, it's not really useful anymore.
Fullack. Even better: the current test runner even lets you avoid the test_suite function in many cases. -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On 07/29/2010 02:17 PM, Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 7/29/10 14:00 , Martin Aspeli wrote:
On 29 July 2010 19:26, yuppie<y.2010@wcm-solutions.de> wrote:
1.) Is it still policy to add these lines?
2.) Is there a better solution for using zope testrunner than the one shown above?
I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically.
Same here. I have never in my life run tests by executing the module. The options provided by the testrunner to pare down what I want to test have always been sufficient for me.
I have had a tendency to add these blurbs myself, thinking "I guess it's useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, it's not really useful anymore.
Fullack. Even better: the current test runner even lets you avoid the test_suite function in many cases. -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
participants (9)
-
Charlie Clark -
Christian Theune -
Hanno Schlichting -
Jens Vagelpohl -
Marius Gedminas -
Martin Aspeli -
Tres Seaver -
Wichert Akkerman -
yuppie