New subject: [Zope-dev] The Debugger Is Your Friend - an object Browser would be nice too

28 Feb 2000

      And this document tells you why!  Last week James W. Howe expressed pain
in now knowing how to debug Zope, or even if it is possible.  Well, it
is, there is even a Zope debugger!  But, alas, until now it was
undocumented.

So here it is folks, 'The Debugger Is Your Friend'.  This is a FIRST
DRAFT so please read it and send your comments and criticisms to me.

The Debugger Is Your Friend

  Jim Fulton is fond of the expression 'The debugger is your friend'.
  And although debugging Zope can sometimes be a bit painful, once you
  are used to the rythum the debugger truly is your friend, and will
  save you years off your life.

  Zope has been designed to work in an integrated way with the Python
  debugger (pdb).  In order to really be able to use this document,
  you must know how to use pdb.  pdb is pretty simple as command line
  debuggers go, and anyone familiar with other popular command line
  debuggers (like gdb) will feel right at home in pdb.  If you are
  more familiar with the graphical, bells-and-whistles type debugger,
  I suggest you read the pdb documentation in the standard Python
  module documentation on the "Python website":http://www.python.org/.

  For the purposes of this document, I will refer to the debugger as
  the 'Zope debugger', even though most of it is actually the Python
  debugger.  This is not to dis the python people, it's just a handy
  convention.  Whenever I refer to just the debugger that is not in
  the context of debugging Zope, I will say 'pdb'.

  To debug Zope, you must first shut down the Zope process.  It is not
  possible to debug Zope and run it at the same time, as the debugger
  stops the entire process dead in its tracks (Note: If you are using
  ZEO, it is possible to run Zope and debug it simultaneously).  Also,
  starting up the debugger will by default start Zope in single
  threaded mode.  It is not possible to run the debugger and Zope in
  multi-threaded mode at the same time.  This is normally not an issue
  unless you are doing some pretty complex low level stuff, which you
  shouldn't be messing with anyway if you have to read this, right?

  For most Zope developer's purposes, the debugger is needed to debug
  some sort of application level programming error.  A common scenario
  is when developing a new third party Product for Zope.  Products
  extend Zope's functionality but they also present the same kind of
  debugging problems that are commonly found in any programming
  environment.  It is useful to have an existing debugging
  infrastructure to help you jump immediatly to your new object and
  debug it and play with it directly in pdb.  The Zope debugger lets
  you do this.

  It is also useful to actually debug Zope itself.  For example, if
  you discover some sort of obscure bug in Zope's object request
  broker (ZPublisher) it would be useful to tell Zope to stop at a
  certain point in ZPublisher and bring up the debugger.  The Zope
  debugger also lets you do this.

  In reality, the 'Zope' part of the Zope debugger is actually just a
  handy way to start up Zope with some pre-configured break points and
  to tell the debugger where in Zope you want to start debugging.
  I'll stop talking now and give some examples.

  Remember, for this example to work, you MUST shut down Zope.
  Debugging Zope starts in Zope's 'lib/python' directory.  'lib' is a
  sub-directory of Zope's top level directory.  Go to the 'lib/python'
  directory and fire up Python 1.5.2::

    Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on
win32
    Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
    >>> import Zope, ZPublisher
    >>>

  Here we have run the python interpreter (which is where using the
  debugger takes place) and imported two modules, 'Zope' and
  'ZPublisher'.  If Python complains about an ImportError and not
  being able to find either module, then you are probably in the wrong
  directory, or you have not compiled Zope properly.

  The Zope module is the main Zope application.  Now, in most cases,
  the term 'application' when used in Zope means some sort of
  application developed on top of Zope, but in this case, we mean Zope
  itself is the application.  The ZPublisher module is the Zope 'ORB'
  (object request broker) and is needed to run the debugger.

  If trying to import these modules results in a 'cannot lock
  database' error, then another Zope process is using your database.
  Shut that Zope down if you want to debug it.  If you get error about
  'cannot open file such-and-such' then you do not have proper
  permission to read or write one of Zope's files.  Get the right
  permission and try again.

  At this point, if python did not complain, then all is well and you
  are on your way to debugging Zope.  First, let's try something
  really neat.  Zope is, as you should know, completely protocol
  agnostic.  You do not need a webserver to actually call and use
  Zope, you can do it right here from this python prompt!  To
  illustrate this, let's call your Zope site and ask it for the very
  root level URL.  If your Zope website's URL is
  'http://www.mysite.org/', then the very root level URL is '' (or
  '/').  Calling this object with the debugger is as easy as, and just
  like *calling it through the web*.  The debugger will set up all
  kinds of fake environment to make Zope not know the difference
  between you calling it here with the debugger and accessing your top
  level root object::

    >>> ZPublisher.Zope('')
    Status: 200 OK
    X-Powered-By: Zope (www.zope.org), Python (www.python.org)
    Content-Length: 1238
    Content-Type: text/html

    <HTML><HEAD><TITLE>Zope</TITLE>

      ... blah blah...

    </BODY></HTML>
    >>> 

  Wow, wasn't that cool?  If you look closely, you will see that the
  content returned is *exactly* that which is returned when you call
  your root level object through HTTP, including all the HTTP headers.

  So now let's suppose that you have an object in your Zope's root
  level folder called 'aFoo'.  aFoo is an instance of your Foo class
  You can call your aFoo instance with the debugger like so::

    >>> ZPublisher.Zope('aFoo')

  (or you could say '/aFoo' or '/aFoo/' etc..., the rules are the same
  as HTTP).  This is a handy way to test out your objects without ever
  leaving python.

  Now, let's get to actually debugging your object.  Debugging,
  typically, involves running your 'program' up to a point where you
  think it's failing, and then inspecting the state of your variables
  and objects.  The easy part is the actual inspection, the hard part
  is getting your program to stop at the right point.  For example, if
  you suspect your logic is failing right at the one hundredth
  iteration in a large for loop, then you need to cleverly construct a
  breakpoint to stop your program right before this iteration.

  The actual techniques of doing things like this, conditional
  breakpoints and other advanced bebugging, is beyond the scope of
  this document.  Many debugger come with documentation, but few come
  with an explanation of the 'Zen' of debugging.  Your best bet is to
  play with the debugger as much as possible, and to even use it to
  help you activly engineer your software.

  So, for the sake or example, let's say that your 'aFoo' object is
  defined in a Zope Product called 'FooProduct', and is located in the
  'lib/python/Products/FooProduct' directory.  The class that defines
  the 'aFoo' instance is also called 'aFoo', and is defined in the
  'aFoo.py' module in your Product.  Therefore, from Zope's
  perspective, your aFoo's classes fully qualified name is
  'Products.FooProduct.aFoo.aFoo'.  All Zope objects have this
  kind of fully qualified name.  For example, the ZCatalog class can
  be found in 'Products.ZCatalog.ZCatalog.ZCatalog (The redundancy is
  because the Product, Module, and class are all named 'ZCatalog').

  Now, your 'aFoo' class defines a method called 'fooMethod' that is
  definatly not doing the right thing for you.  Here is a possible
  definition of your class and method::

    class aFoo:

      def fooMethod(self, N):
        """ return the sum from 1 to N """
        sum = 0
        for x in range(N):
          sum = sum + x
        return sum

  This method can be called with N=5 through the web by visiting
  'http://www.mysite.org/aFoo/fooMethod?N=5'.  This will make Zope
  return the value '15', renderered into an HTML document.  There's
  actually nothing wrong with this method to debug, but for the sake
  of argument let's suppose that returning the sumation of N is not
  what we wanted, and therefore we want to debug this method.

  Well then, let's fire up the debugger!  This is done in a very
  similar way to how we called Zope through the python interpreter
  before, except that we introduce one new argument to the call to
  'Zope'::

    >>> ZPublisher.Zope('/aFoo/fooMethod?N=5', d=1)
    * Type "s<cr>c<cr>" to jump to beginning of real publishing process.
    * Then type c<cr> to jump to the beginning of the URL traversal
      algorithm.
    * Then type c<cr> to jump to published object call.
    > <string>(0)?()
    pdb> 

  Here, we are calling Zope from the interpreter, just like before,
  but there are two differences.  First, we are specificaly calling
  our method ('fooMethod') with an argument, and second, we have
  provided a new argument to the Zope call, 'd=1'.  The 'd' argument,
  when true, causes Zope to fire up in the python debugger, pdb.
  Notice how the python prompt changed from '>>>' to 'pdb>'.  This
  indicates that we are in the debugger.

  When you first fire up the debugger, Zope gives you a helpfull
  message that tells you how to get to your object.  To understand
  this message, it's useful to know how we have set Zope up to be
  debugged.  When Zope fires up in debugger mode, there are three
  breakpoints set for you automatically (if you don't know what a
  breakpoint is, you need to read the python debugger documentation).
  The first breakpoint stops the program at the point that ZPublisher
  (the Zope ORB) tries to publish the application module (in this
  case, the application module is 'Zope').  The second breakpoint
  stops the program right before ZPublisher tries to traverse down the
  provided URL path (in this case, '/aFoo/fooMethod').  The third
  breakpoint will stop the program right before ZPublisher calls the
  object it finds that matches the URL path (in this case, the 'aFoo'
  object).

  So, the little blurb that comes up and tells you some keys to press
  is telling you these things in a terse way.  Hitting 's' will 's'tep
  you into the debugger, and hitting 'c' will 'c'ontinue the execution
  of the program until it hits a breakpoint.

  Note however that none of these breakpoints will stop the program at
  'fooMethod'.  To stop the debugger right there, you need to tell the
  debugger to set a new breakpoint.  This is done quite easily.
  First, you must import your Python module (in this case, a Zope
  Product called 'aFoo', note the above discussion on 'fully
  qualified' names).

    pdb> import Products
    pdb> b Products.FooProduct.aFoo.aFoo.fooMethod
    Breakpoint 5 at C:\Program
Files\WebSite\lib\python\Products\FooProduct\aFoo.py:42
    pdb> 

  First, we import 'Products'.  Since your module is a Zope Products,
  it can be found in the Products Module.  Next, we set a new
  breakpoint with the 'b'reak debugger command (pdb allows you to use
  single letter commands, but you could have also used the entire word
  'break').  The breakpoint we set is
  'Products.FooProduct.aFoo.aFoo.fooMethod'.  After setting this
  breakpoint, the debugger will respond that it found the method in
  question in a certain file, on a certain line (in this case, the
  ficticious line 42) and return you to the debugger.

  Whew!  Lots of stuff to take in there in a few short paragraphs, but
  you are now on your way to debugging your method with style.  Now,
  we want to get to our 'fooMethod' so we can start debugging it.
  But along the way, we must first 'c'ontinue through the various
  breakpoints that Zope has set for us.  Although this may seem like a
  bit of a burden, it's actually quite good to get a feel for how Zope
  works internally by getting down the rythum that Zope uses to
  publish your object.  In these next examples, my inline comments
  will begin with '#".  Obviously, you won't see these comments when
  you are debugging.  So let's debug::

    pdb> s
    # 's'tep into the actual debuging

    > <string>(1)?()
    # this is pdb's response to being steped into, ignore it

    pdb> c
    # now, let's 'c'ontinue onto the next breakpoint

    > C:\Program
Files\WebSite\lib\python\ZPublisher\Publish.py(112)publish()
    -> def publish(request, module_name, after_list, debug=0,

    # pdb has stoped at the first breakpoint, which is the point where
    # ZPubisher tries to publish the application module.

    pdb> c
    # continuing onto the next breakpoint we get...

    > C:\Program
Files\WebSite\lib\python\ZPublisher\Publish.py(101)call_object()
    -> def call_object(object, args, request):

    # Here, ZPublisher (which is now publishing the application) has
    # found your object and is about to call it.  Let's jump out here
    # and discuss this point...

    # 'Calling' your object consists of applying the arguments supplied
by
    # ZPublisher against the object.  Here, you can see how ZPublisher
is
    # passing three arguments into this process.  The first argument is
    # 'object' and is the actual object you want to call.  This can be
    # verified by 'p'rinting the object::

    pdb> p object
    <aFoo instance at 00AFE410>

    # Neat!  So here you can inspect your object (with the 'p'rint
    # command) and even play with it a bit, but we're not there yet!
    #
    # The next argument is 'args'.  This is a tuple of arguments that
    # ZPublisher will apply against your object call.
    #
    # The final argument is 'request'.  This is the 'request object' and
    # will eventually be transformed in to the DTML usable object
'REQUEST'.

    pdb> c    

    # Now let's continue, which breakpoint is next?  Yours!

    > C:\Program
Files\WebSite\lib\python\Products\FooProduct\aFoo.py(42)fooMethod()
    -> def fooMethod(self, N)

    # and now we are here, at YOUR method.  How can we be sure, well,
    # let's tell the debugger to show us where we are in the code:

    pdb> l
    41      def fooMethod(self, N):
    42        """ return the sum from 1 to N """
    43->      sum = 0
    44        for x in range(N):
    45          sum = sum + x
    46        return sum

  And that's it, really.  From here, with a little knowledge of the
  python debugger, you should be able to do any kind of debugging task
  that is needed.

The Debugger Is Your Friend

Michel Pelletier

morton＠dennisinter.com

Loren Stafford

Rik Hoekstra

tags

participants (4)