And this document tells you why! Last week James W. Howe expressed pain in now knowing how to debug Zope, or even if it is possible. Well, it is, there is even a Zope debugger! But, alas, until now it was undocumented. So here it is folks, 'The Debugger Is Your Friend'. This is a FIRST DRAFT so please read it and send your comments and criticisms to me. The Debugger Is Your Friend Jim Fulton is fond of the expression 'The debugger is your friend'. And although debugging Zope can sometimes be a bit painful, once you are used to the rythum the debugger truly is your friend, and will save you years off your life. Zope has been designed to work in an integrated way with the Python debugger (pdb). In order to really be able to use this document, you must know how to use pdb. pdb is pretty simple as command line debuggers go, and anyone familiar with other popular command line debuggers (like gdb) will feel right at home in pdb. If you are more familiar with the graphical, bells-and-whistles type debugger, I suggest you read the pdb documentation in the standard Python module documentation on the "Python website":http://www.python.org/. For the purposes of this document, I will refer to the debugger as the 'Zope debugger', even though most of it is actually the Python debugger. This is not to dis the python people, it's just a handy convention. Whenever I refer to just the debugger that is not in the context of debugging Zope, I will say 'pdb'. To debug Zope, you must first shut down the Zope process. It is not possible to debug Zope and run it at the same time, as the debugger stops the entire process dead in its tracks (Note: If you are using ZEO, it is possible to run Zope and debug it simultaneously). Also, starting up the debugger will by default start Zope in single threaded mode. It is not possible to run the debugger and Zope in multi-threaded mode at the same time. This is normally not an issue unless you are doing some pretty complex low level stuff, which you shouldn't be messing with anyway if you have to read this, right? For most Zope developer's purposes, the debugger is needed to debug some sort of application level programming error. A common scenario is when developing a new third party Product for Zope. Products extend Zope's functionality but they also present the same kind of debugging problems that are commonly found in any programming environment. It is useful to have an existing debugging infrastructure to help you jump immediatly to your new object and debug it and play with it directly in pdb. The Zope debugger lets you do this. It is also useful to actually debug Zope itself. For example, if you discover some sort of obscure bug in Zope's object request broker (ZPublisher) it would be useful to tell Zope to stop at a certain point in ZPublisher and bring up the debugger. The Zope debugger also lets you do this. In reality, the 'Zope' part of the Zope debugger is actually just a handy way to start up Zope with some pre-configured break points and to tell the debugger where in Zope you want to start debugging. I'll stop talking now and give some examples. Remember, for this example to work, you MUST shut down Zope. Debugging Zope starts in Zope's 'lib/python' directory. 'lib' is a sub-directory of Zope's top level directory. Go to the 'lib/python' directory and fire up Python 1.5.2:: Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import Zope, ZPublisher >>> Here we have run the python interpreter (which is where using the debugger takes place) and imported two modules, 'Zope' and 'ZPublisher'. If Python complains about an ImportError and not being able to find either module, then you are probably in the wrong directory, or you have not compiled Zope properly. The Zope module is the main Zope application. Now, in most cases, the term 'application' when used in Zope means some sort of application developed on top of Zope, but in this case, we mean Zope itself is the application. The ZPublisher module is the Zope 'ORB' (object request broker) and is needed to run the debugger. If trying to import these modules results in a 'cannot lock database' error, then another Zope process is using your database. Shut that Zope down if you want to debug it. If you get error about 'cannot open file such-and-such' then you do not have proper permission to read or write one of Zope's files. Get the right permission and try again. At this point, if python did not complain, then all is well and you are on your way to debugging Zope. First, let's try something really neat. Zope is, as you should know, completely protocol agnostic. You do not need a webserver to actually call and use Zope, you can do it right here from this python prompt! To illustrate this, let's call your Zope site and ask it for the very root level URL. If your Zope website's URL is 'http://www.mysite.org/', then the very root level URL is '' (or '/'). Calling this object with the debugger is as easy as, and just like *calling it through the web*. The debugger will set up all kinds of fake environment to make Zope not know the difference between you calling it here with the debugger and accessing your top level root object:: >>> ZPublisher.Zope('') Status: 200 OK X-Powered-By: Zope (www.zope.org), Python (www.python.org) Content-Length: 1238 Content-Type: text/html <HTML><HEAD><TITLE>Zope</TITLE> ... blah blah... </BODY></HTML> >>> Wow, wasn't that cool? If you look closely, you will see that the content returned is *exactly* that which is returned when you call your root level object through HTTP, including all the HTTP headers. So now let's suppose that you have an object in your Zope's root level folder called 'aFoo'. aFoo is an instance of your Foo class You can call your aFoo instance with the debugger like so:: >>> ZPublisher.Zope('aFoo') (or you could say '/aFoo' or '/aFoo/' etc..., the rules are the same as HTTP). This is a handy way to test out your objects without ever leaving python. Now, let's get to actually debugging your object. Debugging, typically, involves running your 'program' up to a point where you think it's failing, and then inspecting the state of your variables and objects. The easy part is the actual inspection, the hard part is getting your program to stop at the right point. For example, if you suspect your logic is failing right at the one hundredth iteration in a large for loop, then you need to cleverly construct a breakpoint to stop your program right before this iteration. The actual techniques of doing things like this, conditional breakpoints and other advanced bebugging, is beyond the scope of this document. Many debugger come with documentation, but few come with an explanation of the 'Zen' of debugging. Your best bet is to play with the debugger as much as possible, and to even use it to help you activly engineer your software. So, for the sake or example, let's say that your 'aFoo' object is defined in a Zope Product called 'FooProduct', and is located in the 'lib/python/Products/FooProduct' directory. The class that defines the 'aFoo' instance is also called 'aFoo', and is defined in the 'aFoo.py' module in your Product. Therefore, from Zope's perspective, your aFoo's classes fully qualified name is 'Products.FooProduct.aFoo.aFoo'. All Zope objects have this kind of fully qualified name. For example, the ZCatalog class can be found in 'Products.ZCatalog.ZCatalog.ZCatalog (The redundancy is because the Product, Module, and class are all named 'ZCatalog'). Now, your 'aFoo' class defines a method called 'fooMethod' that is definatly not doing the right thing for you. Here is a possible definition of your class and method:: class aFoo: def fooMethod(self, N): """ return the sum from 1 to N """ sum = 0 for x in range(N): sum = sum + x return sum This method can be called with N=5 through the web by visiting 'http://www.mysite.org/aFoo/fooMethod?N=5'. This will make Zope return the value '15', renderered into an HTML document. There's actually nothing wrong with this method to debug, but for the sake of argument let's suppose that returning the sumation of N is not what we wanted, and therefore we want to debug this method. Well then, let's fire up the debugger! This is done in a very similar way to how we called Zope through the python interpreter before, except that we introduce one new argument to the call to 'Zope':: >>> ZPublisher.Zope('/aFoo/fooMethod?N=5', d=1) * Type "s<cr>c<cr>" to jump to beginning of real publishing process. * Then type c<cr> to jump to the beginning of the URL traversal algorithm. * Then type c<cr> to jump to published object call. > <string>(0)?() pdb> Here, we are calling Zope from the interpreter, just like before, but there are two differences. First, we are specificaly calling our method ('fooMethod') with an argument, and second, we have provided a new argument to the Zope call, 'd=1'. The 'd' argument, when true, causes Zope to fire up in the python debugger, pdb. Notice how the python prompt changed from '>>>' to 'pdb>'. This indicates that we are in the debugger. When you first fire up the debugger, Zope gives you a helpfull message that tells you how to get to your object. To understand this message, it's useful to know how we have set Zope up to be debugged. When Zope fires up in debugger mode, there are three breakpoints set for you automatically (if you don't know what a breakpoint is, you need to read the python debugger documentation). The first breakpoint stops the program at the point that ZPublisher (the Zope ORB) tries to publish the application module (in this case, the application module is 'Zope'). The second breakpoint stops the program right before ZPublisher tries to traverse down the provided URL path (in this case, '/aFoo/fooMethod'). The third breakpoint will stop the program right before ZPublisher calls the object it finds that matches the URL path (in this case, the 'aFoo' object). So, the little blurb that comes up and tells you some keys to press is telling you these things in a terse way. Hitting 's' will 's'tep you into the debugger, and hitting 'c' will 'c'ontinue the execution of the program until it hits a breakpoint. Note however that none of these breakpoints will stop the program at 'fooMethod'. To stop the debugger right there, you need to tell the debugger to set a new breakpoint. This is done quite easily. First, you must import your Python module (in this case, a Zope Product called 'aFoo', note the above discussion on 'fully qualified' names). pdb> import Products pdb> b Products.FooProduct.aFoo.aFoo.fooMethod Breakpoint 5 at C:\Program Files\WebSite\lib\python\Products\FooProduct\aFoo.py:42 pdb> First, we import 'Products'. Since your module is a Zope Products, it can be found in the Products Module. Next, we set a new breakpoint with the 'b'reak debugger command (pdb allows you to use single letter commands, but you could have also used the entire word 'break'). The breakpoint we set is 'Products.FooProduct.aFoo.aFoo.fooMethod'. After setting this breakpoint, the debugger will respond that it found the method in question in a certain file, on a certain line (in this case, the ficticious line 42) and return you to the debugger. Whew! Lots of stuff to take in there in a few short paragraphs, but you are now on your way to debugging your method with style. Now, we want to get to our 'fooMethod' so we can start debugging it. But along the way, we must first 'c'ontinue through the various breakpoints that Zope has set for us. Although this may seem like a bit of a burden, it's actually quite good to get a feel for how Zope works internally by getting down the rythum that Zope uses to publish your object. In these next examples, my inline comments will begin with '#". Obviously, you won't see these comments when you are debugging. So let's debug:: pdb> s # 's'tep into the actual debuging > <string>(1)?() # this is pdb's response to being steped into, ignore it pdb> c # now, let's 'c'ontinue onto the next breakpoint > C:\Program Files\WebSite\lib\python\ZPublisher\Publish.py(112)publish() -> def publish(request, module_name, after_list, debug=0, # pdb has stoped at the first breakpoint, which is the point where # ZPubisher tries to publish the application module. pdb> c # continuing onto the next breakpoint we get... > C:\Program Files\WebSite\lib\python\ZPublisher\Publish.py(101)call_object() -> def call_object(object, args, request): # Here, ZPublisher (which is now publishing the application) has # found your object and is about to call it. Let's jump out here # and discuss this point... # 'Calling' your object consists of applying the arguments supplied by # ZPublisher against the object. Here, you can see how ZPublisher is # passing three arguments into this process. The first argument is # 'object' and is the actual object you want to call. This can be # verified by 'p'rinting the object:: pdb> p object <aFoo instance at 00AFE410> # Neat! So here you can inspect your object (with the 'p'rint # command) and even play with it a bit, but we're not there yet! # # The next argument is 'args'. This is a tuple of arguments that # ZPublisher will apply against your object call. # # The final argument is 'request'. This is the 'request object' and # will eventually be transformed in to the DTML usable object 'REQUEST'. pdb> c # Now let's continue, which breakpoint is next? Yours! > C:\Program Files\WebSite\lib\python\Products\FooProduct\aFoo.py(42)fooMethod() -> def fooMethod(self, N) # and now we are here, at YOUR method. How can we be sure, well, # let's tell the debugger to show us where we are in the code: pdb> l 41 def fooMethod(self, N): 42 """ return the sum from 1 to N """ 43-> sum = 0 44 for x in range(N): 45 sum = sum + x 46 return sum And that's it, really. From here, with a little knowledge of the python debugger, you should be able to do any kind of debugging task that is needed.