zope.testing.testrunner.debug.post_mortem and try/finally
When a try/finally clause is (appropriately) used to do cleanup after an exception during a test run, it often tears down parts of the fixture that are needed in order to do useful post_mortem debugging of the exception, such as closing the request or db connections. What is the best way to do post_mortem debugging with the stack in the state it was at the time of the exception? For a while now, I've been repeatedly modifying eggs in my development environment at the relevant try/finally clauses to invoke post_mortem before the tear down is done, sub-optimal to say the least. :) I find myself doing it often its time to invest in a better way. Is there some Python voodoo I'm unaware of to get a post_mortem to reflect the stack before try/finally clauses? If not, is putting some sort of hook into the relevant try/finally clauses the best way to address this? If not, what should I be doing? If putting hooks into the try/finally clauses is the right way, then it would be nice to have a somewhat canonical way to do this. It would also be nice to have a way to pass something down the line so that post_mortem() only gets called once per exception. Thoughts, Ross
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ross Patterson wrote:
When a try/finally clause is (appropriately) used to do cleanup after an exception during a test run, it often tears down parts of the fixture that are needed in order to do useful post_mortem debugging of the exception, such as closing the request or db connections. What is the best way to do post_mortem debugging with the stack in the state it was at the time of the exception?
For a while now, I've been repeatedly modifying eggs in my development environment at the relevant try/finally clauses to invoke post_mortem before the tear down is done, sub-optimal to say the least. :) I find myself doing it often its time to invest in a better way.
Is there some Python voodoo I'm unaware of to get a post_mortem to reflect the stack before try/finally clauses? If not, is putting some sort of hook into the relevant try/finally clauses the best way to address this? If not, what should I be doing?
If putting hooks into the try/finally clauses is the right way, then it would be nice to have a somewhat canonical way to do this. It would also be nice to have a way to pass something down the line so that post_mortem() only gets called once per exception.
Are you using try:...finally:... inside your testcase methods? If so, why not just move the cleanup invocation into your 'tearDown' for the testcase class: at that point, the '-D' option to the testrunner will stop you at the error, with the tearDown not yet called. Or are you doing this in doctests? If so, move the code you are testing into a real testing framework. ;) Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksWrVkACgkQ+gerLs4ltQ4JbQCeOuA9k90+E/KF2HRTO1fy46B/ tPYAoJ3CKOfxu+Ty7a5Tystu/PL5iy9O =cSLA -----END PGP SIGNATURE-----
Tres Seaver <tseaver@palladion.com> writes:
Ross Patterson wrote:
When a try/finally clause is (appropriately) used to do cleanup after an exception during a test run, it often tears down parts of the fixture that are needed in order to do useful post_mortem debugging of the exception, such as closing the request or db connections. What is the best way to do post_mortem debugging with the stack in the state it was
at the time of the exception?
For a while now, I've been repeatedly modifying eggs in my development environment at the relevant try/finally clauses to invoke post_mortem before the tear down is done, sub-optimal to say the least. :) I find myself doing it often its time to invest in a better way.
Is there some Python voodoo I'm unaware of to get a post_mortem to reflect the stack before try/finally clauses? If not, is putting some sort of hook into the relevant try/finally clauses the best way to address this? If not, what should I be doing?
If putting hooks into the try/finally clauses is the right way, then it would be nice to have a somewhat canonical way to do this. It would also be nice to have a way to pass something down the line so that post_mortem() only gets called once per exception.
Are you using try:...finally:... inside your testcase methods? If so, why not just move the cleanup invocation into your 'tearDown' for the testcase class: at that point, the '-D' option to the testrunner will stop you at the error, with the tearDown not yet called.
I'm sorry, I was unclear, the try/finally clauses are not necessarily in *test tearDown* methods (though I used that language), they are often a part of the application being tested, such as closing the request, closing DB connections, or aborting transactions.
Or are you doing this in doctests? If so, move the code you are testing into a real testing framework. ;)
Oh, goody. Derogatory evangelism. :) Doctests have tearDown just like other tests. Ross
On Wed, Dec 2, 2009 at 1:22 PM, Ross Patterson <me@rpatterson.net> wrote:
I'm sorry, I was unclear, the try/finally clauses are not necessarily in *test tearDown* methods (though I used that language), they are often a part of the application being tested, such as closing the request, closing DB connections, or aborting transactions.
In those situations I just edit the code and put a "import pdb;pdb.set_trace()" at the top of the finally: block and re-run the test. It is an interesting idea to provide some way to make that automatic. Maybe a settrace hook that will invoke pdb at the top of every finally clause that is encountered, somewhat similar to the -D option. You might end up getting way too many false positives though. Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so. -- Benji York
Benji York <benji@zope.com> writes:
On Wed, Dec 2, 2009 at 1:22 PM, Ross Patterson <me@rpatterson.net> wrote:
I'm sorry, I was unclear, the try/finally clauses are not necessarily in *test tearDown* methods (though I used that language), they are often a part of the application being tested, such as closing the request, closing DB connections, or aborting transactions.
In those situations I just edit the code and put a "import pdb;pdb.set_trace()" at the top of the finally: block and re-run the test.
It is an interesting idea to provide some way to make that automatic. Maybe a settrace hook that will invoke pdb at the top of every finally clause that is encountered, somewhat similar to the -D option. You might end up getting way too many false positives though.
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Or maybe a combination of the two where a configuration file is read for file and line numbers at which to do a post_mortem using a settrace hook where if the first one of those is hit no further post_mortem's are executed? Ross
On Wed, Dec 02, 2009 at 01:36:37PM -0500, Benji York wrote:
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Is that possible? I once spent hours studying pdb docs and found no way to set a breakpoint in advance, without modifying the source file in question, and without having the user to do manual interaction at the beginning. The closest that I can come is import pdb dbg = pdb.Pdb() dbg.set_break(filename, lineno) dbg.runcall(entry_point) but this breaks into the pdb prompt at once, without waiting for the breakpoint to get triggered, and I have to manually 'c' to get unstuck. What am I missing? Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
On Wed, Dec 2, 2009 at 5:06 PM, Marius Gedminas <marius@gedmin.as> wrote:
On Wed, Dec 02, 2009 at 01:36:37PM -0500, Benji York wrote:
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Is that possible? I once spent hours studying pdb docs and found no way to set a breakpoint in advance, without modifying the source file in question, and without having the user to do manual interaction at the beginning.
I was envisioning using sys.settrace() to set a trace function that would notice when the desired line of code is being executed and then trigger pdb. I didn't plan on pdb doing the breakpoint handling itself. -- Benji York
On 2009-12-2 23:06, Marius Gedminas wrote:
On Wed, Dec 02, 2009 at 01:36:37PM -0500, Benji York wrote:
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Is that possible? I once spent hours studying pdb docs and found no way to set a breakpoint in advance, without modifying the source file in question, and without having the user to do manual interaction at the beginning.
I think mr.freeze (see http://pypi.python.org/pypi/mr.freeze ) can do this. Perhaps there is some code there you could borrow. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
On Thu, Dec 03, 2009 at 08:55:32AM +0100, Wichert Akkerman wrote:
On 2009-12-2 23:06, Marius Gedminas wrote:
On Wed, Dec 02, 2009 at 01:36:37PM -0500, Benji York wrote:
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Is that possible? I once spent hours studying pdb docs and found no way to set a breakpoint in advance, without modifying the source file in question, and without having the user to do manual interaction at the beginning.
I think mr.freeze (see http://pypi.python.org/pypi/mr.freeze ) can do this.
Thank you for the link, this looks very interesting.
Perhaps there is some code there you could borrow.
Or just depend and invoke it. AFAICS mr.freeze subclasses Pdb and rewires some of its internals in a nontrivial fashion to be able to do that: http://dev.plone.org/collective/browser/mr.freeze/trunk/mr/freeze/freeze.py Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
On Thu, Dec 03, 2009 at 03:51:06PM +0200, Marius Gedminas wrote:
On Thu, Dec 03, 2009 at 08:55:32AM +0100, Wichert Akkerman wrote:
On 2009-12-2 23:06, Marius Gedminas wrote:
On Wed, Dec 02, 2009 at 01:36:37PM -0500, Benji York wrote:
Here's another idea: a testrunner option that takes a file name and line number and inserts a breakpoint at that position. That way you can get the same effect as editing the code without actually having to do so.
Is that possible? I once spent hours studying pdb docs and found no way to set a breakpoint in advance, without modifying the source file in question, and without having the user to do manual interaction at the beginning.
I think mr.freeze (see http://pypi.python.org/pypi/mr.freeze ) can do this.
Thank you for the link, this looks very interesting.
Perhaps there is some code there you could borrow.
Or just depend and invoke it.
Better not, it pulls in too much stuff (WSGI and Zope 2 bits).
AFAICS mr.freeze subclasses Pdb and rewires some of its internals in a nontrivial fashion to be able to do that: http://dev.plone.org/collective/browser/mr.freeze/trunk/mr/freeze/freeze.py
I've had some success with this: import pdb class SlightlyLessInsanePdb(pdb.Pdb): first_time = True def user_line(self, *a, **kw): if self.first_time: self.set_continue() self.first_time = False else: pdb.Pdb.user_line(self, *a, **kw) def run_with_breakpoint(filename, lineno, func, *args, **kw): dbg = SlightlyLessInsanePdb() dbg.set_break(filename, lineno) return dbg.runcall(func, *args, **kw) Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
On Thu, Dec 3, 2009 at 9:43 AM, Marius Gedminas <marius@gedmin.as> wrote:
I've had some success with this:
[snip code] Nice. I'd love to see this wired into the testrunner so people could specify breakpoints on the command line while running tests. If no one else gets around to that, I might. One day. -- Benji York
On 12/03/2009 04:08 PM, Benji York wrote:
On Thu, Dec 3, 2009 at 9:43 AM, Marius Gedminas<marius@gedmin.as> wrote:
I've had some success with this:
[snip code]
Nice. I'd love to see this wired into the testrunner so people could specify breakpoints on the command line while running tests.
If no one else gets around to that, I might. One day.
Well, if someone threw this code in the bug tracker on the wish list, then it's even more likely someone will get around to do it. Some day. ;) -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ross Patterson wrote:
Tres Seaver <tseaver@palladion.com> writes:
Are you using try:...finally:... inside your testcase methods? If so, why not just move the cleanup invocation into your 'tearDown' for the testcase class: at that point, the '-D' option to the testrunner will stop you at the error, with the tearDown not yet called.
I'm sorry, I was unclear, the try/finally clauses are not necessarily in *test tearDown* methods (though I used that language), they are often a part of the application being tested, such as closing the request, closing DB connections, or aborting transactions.
OK. In such cases, I usually add a 'pdb.set_trace' to the testcase just before calling into the application-under-test, and step in from there.
Or are you doing this in doctests? If so, move the code you are testing into a real testing framework. ;)
Oh, goody. Derogatory evangelism. :) Doctests have tearDown just like other tests.
The teardown for doctests runs at the end of the entire text file, which is far too late for the kind of granular teardown needed to maintain proper test isolation. Because of this granularity, people sometimes inline 'try:...finally:...' into the body of the tests, trying to achieve some isolation / cleanliness. Doctests don't work for thorough unit testing: trying to make them substitue for unit tests just ruins any value they have as documentation. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksWu+0ACgkQ+gerLs4ltQ5VZgCg0PljroCmLWSO/jjx7dUTu4sj Af8AoID1nhnnzzqtvqNZUwo5FO4YmPtU =4Zx2 -----END PGP SIGNATURE-----
On Wed, Dec 02, 2009 at 09:08:51AM -0800, Ross Patterson wrote:
When a try/finally clause is (appropriately) used to do cleanup after an exception during a test run, it often tears down parts of the fixture that are needed in order to do useful post_mortem debugging of the exception, such as closing the request or db connections. What is the best way to do post_mortem debugging with the stack in the state it was at the time of the exception?
What I always do is find the line of code that raises the exception and wrap it with try: ... except: import pdb; pdb.set_trace() which is more or less exactly what you do:
For a while now, I've been repeatedly modifying eggs in my development environment at the relevant try/finally clauses to invoke post_mortem before the tear down is done, sub-optimal to say the least. :) I find myself doing it often its time to invest in a better way.
Worst part is when you use a shared buildout egg cache and forget to undo your experimental changes. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
participants (6)
-
Benji York -
Christian Theune -
Marius Gedminas -
Ross Patterson -
Tres Seaver -
Wichert Akkerman