[Zope-dev] The Debugger Is Your Friend
Michel Pelletier
michel@digicool.com
Mon, 28 Feb 2000 11:24:05 -0800
And this document tells you why! Last week James W. Howe expressed pain
in now knowing how to debug Zope, or even if it is possible. Well, it
is, there is even a Zope debugger! But, alas, until now it was
undocumented.
So here it is folks, 'The Debugger Is Your Friend'. This is a FIRST
DRAFT so please read it and send your comments and criticisms to me.
The Debugger Is Your Friend
Jim Fulton is fond of the expression 'The debugger is your friend'.
And although debugging Zope can sometimes be a bit painful, once you
are used to the rythum the debugger truly is your friend, and will
save you years off your life.
Zope has been designed to work in an integrated way with the Python
debugger (pdb). In order to really be able to use this document,
you must know how to use pdb. pdb is pretty simple as command line
debuggers go, and anyone familiar with other popular command line
debuggers (like gdb) will feel right at home in pdb. If you are
more familiar with the graphical, bells-and-whistles type debugger,
I suggest you read the pdb documentation in the standard Python
module documentation on the "Python website":http://www.python.org/.
For the purposes of this document, I will refer to the debugger as
the 'Zope debugger', even though most of it is actually the Python
debugger. This is not to dis the python people, it's just a handy
convention. Whenever I refer to just the debugger that is not in
the context of debugging Zope, I will say 'pdb'.
To debug Zope, you must first shut down the Zope process. It is not
possible to debug Zope and run it at the same time, as the debugger
stops the entire process dead in its tracks (Note: If you are using
ZEO, it is possible to run Zope and debug it simultaneously). Also,
starting up the debugger will by default start Zope in single
threaded mode. It is not possible to run the debugger and Zope in
multi-threaded mode at the same time. This is normally not an issue
unless you are doing some pretty complex low level stuff, which you
shouldn't be messing with anyway if you have to read this, right?
For most Zope developer's purposes, the debugger is needed to debug
some sort of application level programming error. A common scenario
is when developing a new third party Product for Zope. Products
extend Zope's functionality but they also present the same kind of
debugging problems that are commonly found in any programming
environment. It is useful to have an existing debugging
infrastructure to help you jump immediatly to your new object and
debug it and play with it directly in pdb. The Zope debugger lets
you do this.
It is also useful to actually debug Zope itself. For example, if
you discover some sort of obscure bug in Zope's object request
broker (ZPublisher) it would be useful to tell Zope to stop at a
certain point in ZPublisher and bring up the debugger. The Zope
debugger also lets you do this.
In reality, the 'Zope' part of the Zope debugger is actually just a
handy way to start up Zope with some pre-configured break points and
to tell the debugger where in Zope you want to start debugging.
I'll stop talking now and give some examples.
Remember, for this example to work, you MUST shut down Zope.
Debugging Zope starts in Zope's 'lib/python' directory. 'lib' is a
sub-directory of Zope's top level directory. Go to the 'lib/python'
directory and fire up Python 1.5.2::
Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on
win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Zope, ZPublisher
>>>
Here we have run the python interpreter (which is where using the
debugger takes place) and imported two modules, 'Zope' and
'ZPublisher'. If Python complains about an ImportError and not
being able to find either module, then you are probably in the wrong
directory, or you have not compiled Zope properly.
The Zope module is the main Zope application. Now, in most cases,
the term 'application' when used in Zope means some sort of
application developed on top of Zope, but in this case, we mean Zope
itself is the application. The ZPublisher module is the Zope 'ORB'
(object request broker) and is needed to run the debugger.
If trying to import these modules results in a 'cannot lock
database' error, then another Zope process is using your database.
Shut that Zope down if you want to debug it. If you get error about
'cannot open file such-and-such' then you do not have proper
permission to read or write one of Zope's files. Get the right
permission and try again.
At this point, if python did not complain, then all is well and you
are on your way to debugging Zope. First, let's try something
really neat. Zope is, as you should know, completely protocol
agnostic. You do not need a webserver to actually call and use
Zope, you can do it right here from this python prompt! To
illustrate this, let's call your Zope site and ask it for the very
root level URL. If your Zope website's URL is
'http://www.mysite.org/', then the very root level URL is '' (or
'/'). Calling this object with the debugger is as easy as, and just
like *calling it through the web*. The debugger will set up all
kinds of fake environment to make Zope not know the difference
between you calling it here with the debugger and accessing your top
level root object::
>>> ZPublisher.Zope('')
Status: 200 OK
X-Powered-By: Zope (www.zope.org), Python (www.python.org)
Content-Length: 1238
Content-Type: text/html
<HTML><HEAD><TITLE>Zope</TITLE>
... blah blah...
</BODY></HTML>
>>>
Wow, wasn't that cool? If you look closely, you will see that the
content returned is *exactly* that which is returned when you call
your root level object through HTTP, including all the HTTP headers.
So now let's suppose that you have an object in your Zope's root
level folder called 'aFoo'. aFoo is an instance of your Foo class
You can call your aFoo instance with the debugger like so::
>>> ZPublisher.Zope('aFoo')
(or you could say '/aFoo' or '/aFoo/' etc..., the rules are the same
as HTTP). This is a handy way to test out your objects without ever
leaving python.
Now, let's get to actually debugging your object. Debugging,
typically, involves running your 'program' up to a point where you
think it's failing, and then inspecting the state of your variables
and objects. The easy part is the actual inspection, the hard part
is getting your program to stop at the right point. For example, if
you suspect your logic is failing right at the one hundredth
iteration in a large for loop, then you need to cleverly construct a
breakpoint to stop your program right before this iteration.
The actual techniques of doing things like this, conditional
breakpoints and other advanced bebugging, is beyond the scope of
this document. Many debugger come with documentation, but few come
with an explanation of the 'Zen' of debugging. Your best bet is to
play with the debugger as much as possible, and to even use it to
help you activly engineer your software.
So, for the sake or example, let's say that your 'aFoo' object is
defined in a Zope Product called 'FooProduct', and is located in the
'lib/python/Products/FooProduct' directory. The class that defines
the 'aFoo' instance is also called 'aFoo', and is defined in the
'aFoo.py' module in your Product. Therefore, from Zope's
perspective, your aFoo's classes fully qualified name is
'Products.FooProduct.aFoo.aFoo'. All Zope objects have this
kind of fully qualified name. For example, the ZCatalog class can
be found in 'Products.ZCatalog.ZCatalog.ZCatalog (The redundancy is
because the Product, Module, and class are all named 'ZCatalog').
Now, your 'aFoo' class defines a method called 'fooMethod' that is
definatly not doing the right thing for you. Here is a possible
definition of your class and method::
class aFoo:
def fooMethod(self, N):
""" return the sum from 1 to N """
sum = 0
for x in range(N):
sum = sum + x
return sum
This method can be called with N=5 through the web by visiting
'http://www.mysite.org/aFoo/fooMethod?N=5'. This will make Zope
return the value '15', renderered into an HTML document. There's
actually nothing wrong with this method to debug, but for the sake
of argument let's suppose that returning the sumation of N is not
what we wanted, and therefore we want to debug this method.
Well then, let's fire up the debugger! This is done in a very
similar way to how we called Zope through the python interpreter
before, except that we introduce one new argument to the call to
'Zope'::
>>> ZPublisher.Zope('/aFoo/fooMethod?N=5', d=1)
* Type "s<cr>c<cr>" to jump to beginning of real publishing process.
* Then type c<cr> to jump to the beginning of the URL traversal
algorithm.
* Then type c<cr> to jump to published object call.
> <string>(0)?()
pdb>
Here, we are calling Zope from the interpreter, just like before,
but there are two differences. First, we are specificaly calling
our method ('fooMethod') with an argument, and second, we have
provided a new argument to the Zope call, 'd=1'. The 'd' argument,
when true, causes Zope to fire up in the python debugger, pdb.
Notice how the python prompt changed from '>>>' to 'pdb>'. This
indicates that we are in the debugger.
When you first fire up the debugger, Zope gives you a helpfull
message that tells you how to get to your object. To understand
this message, it's useful to know how we have set Zope up to be
debugged. When Zope fires up in debugger mode, there are three
breakpoints set for you automatically (if you don't know what a
breakpoint is, you need to read the python debugger documentation).
The first breakpoint stops the program at the point that ZPublisher
(the Zope ORB) tries to publish the application module (in this
case, the application module is 'Zope'). The second breakpoint
stops the program right before ZPublisher tries to traverse down the
provided URL path (in this case, '/aFoo/fooMethod'). The third
breakpoint will stop the program right before ZPublisher calls the
object it finds that matches the URL path (in this case, the 'aFoo'
object).
So, the little blurb that comes up and tells you some keys to press
is telling you these things in a terse way. Hitting 's' will 's'tep
you into the debugger, and hitting 'c' will 'c'ontinue the execution
of the program until it hits a breakpoint.
Note however that none of these breakpoints will stop the program at
'fooMethod'. To stop the debugger right there, you need to tell the
debugger to set a new breakpoint. This is done quite easily.
First, you must import your Python module (in this case, a Zope
Product called 'aFoo', note the above discussion on 'fully
qualified' names).
pdb> import Products
pdb> b Products.FooProduct.aFoo.aFoo.fooMethod
Breakpoint 5 at C:\Program
Files\WebSite\lib\python\Products\FooProduct\aFoo.py:42
pdb>
First, we import 'Products'. Since your module is a Zope Products,
it can be found in the Products Module. Next, we set a new
breakpoint with the 'b'reak debugger command (pdb allows you to use
single letter commands, but you could have also used the entire word
'break'). The breakpoint we set is
'Products.FooProduct.aFoo.aFoo.fooMethod'. After setting this
breakpoint, the debugger will respond that it found the method in
question in a certain file, on a certain line (in this case, the
ficticious line 42) and return you to the debugger.
Whew! Lots of stuff to take in there in a few short paragraphs, but
you are now on your way to debugging your method with style. Now,
we want to get to our 'fooMethod' so we can start debugging it.
But along the way, we must first 'c'ontinue through the various
breakpoints that Zope has set for us. Although this may seem like a
bit of a burden, it's actually quite good to get a feel for how Zope
works internally by getting down the rythum that Zope uses to
publish your object. In these next examples, my inline comments
will begin with '#". Obviously, you won't see these comments when
you are debugging. So let's debug::
pdb> s
# 's'tep into the actual debuging
> <string>(1)?()
# this is pdb's response to being steped into, ignore it
pdb> c
# now, let's 'c'ontinue onto the next breakpoint
> C:\Program
Files\WebSite\lib\python\ZPublisher\Publish.py(112)publish()
-> def publish(request, module_name, after_list, debug=0,
# pdb has stoped at the first breakpoint, which is the point where
# ZPubisher tries to publish the application module.
pdb> c
# continuing onto the next breakpoint we get...
> C:\Program
Files\WebSite\lib\python\ZPublisher\Publish.py(101)call_object()
-> def call_object(object, args, request):
# Here, ZPublisher (which is now publishing the application) has
# found your object and is about to call it. Let's jump out here
# and discuss this point...
# 'Calling' your object consists of applying the arguments supplied
by
# ZPublisher against the object. Here, you can see how ZPublisher
is
# passing three arguments into this process. The first argument is
# 'object' and is the actual object you want to call. This can be
# verified by 'p'rinting the object::
pdb> p object
<aFoo instance at 00AFE410>
# Neat! So here you can inspect your object (with the 'p'rint
# command) and even play with it a bit, but we're not there yet!
#
# The next argument is 'args'. This is a tuple of arguments that
# ZPublisher will apply against your object call.
#
# The final argument is 'request'. This is the 'request object' and
# will eventually be transformed in to the DTML usable object
'REQUEST'.
pdb> c
# Now let's continue, which breakpoint is next? Yours!
> C:\Program
Files\WebSite\lib\python\Products\FooProduct\aFoo.py(42)fooMethod()
-> def fooMethod(self, N)
# and now we are here, at YOUR method. How can we be sure, well,
# let's tell the debugger to show us where we are in the code:
pdb> l
41 def fooMethod(self, N):
42 """ return the sum from 1 to N """
43-> sum = 0
44 for x in range(N):
45 sum = sum + x
46 return sum
And that's it, really. From here, with a little knowledge of the
python debugger, you should be able to do any kind of debugging task
that is needed.