[Zope-dev] The Debugger Is Your Friend - an object Browser would be nice too
Damian Morton
morton@dennisinter.com
Mon, 28 Feb 2000 18:26:56 -0500
An extremely usefull adjunct to the debugger would be an object browser for
the running Zope application.
Unfortunately, using the monitor or even the debugger to browse through the
running application can be extremely painful, especialy as many of the
methods and properties of objects arent displayed using dir(object) or
object.__dict__. What is needed is a browser that understands classes,
inheritance and acquisition.
Im looking to find the list of properties associated with the object
StuffMag.
Heres how Im using the monitor to browse the live application:
### lets bring in something to help format our data
>>> from pprint import pprint
### This is the object I want to examine
>>> app.StuffMag
<StuffClass instance at 011FF190>
#### Lets see whats inside this object
>>> pprint(app.StuffMag.__dict__)
{'BabeSection': <StuffSectionClass instance at 01222720>,
'OtherSection': <StuffSectionClass instance at 01222370>,
'StuffSection': <StuffSectionClass instance at 012227B0>,
'__ac_local_roles__': {'damien.morton': ['Owner']},
'_objects': ({'meta_type': 'Section object', 'id': 'BabeSection'},
{'meta_type': 'Section object', 'id': 'StuffSection'},
{'meta_type': 'Section object', 'id': 'OtherSection'},
{'meta_type': 'Section2 object', 'id': 'section2'},
{'meta_type': 'Section3 object', 'id': 'section3'},
{'meta_type': 'Section5 object', 'id': 'section5'},
{'meta_type': 'Section5 object', 'id': 'section5.2'},
{'meta_type': 'Section5 object', 'id': 'mysection5'},
{'meta_type': 'myClass object', 'id': 'myObject'}),
'body': '',
'heading': 'HI THERE!!!',
'id': 'StuffMag',
'myObject': <myClass instance at 01222660>,
'mysection5': <Section5Class instance at 012222A0>,
'section2': <StuffSection2 instance at 012224D0>,
'section3': <section3Class instance at 012225A0>,
'section5': <Section5Class instance at 012218A0>,
'section5.2': <Section5Class instance at 01222400>}
>>> app.StuffMag.propertysheets
<StuffClass_PropertySheetsClass instance at 011FAA50>
### How did I know this object had a propertysheets attribute?
>>> pprint(app.StuffMag.__class__.__dict__)
{'Section5Class': <MWp instance at 011FACE0>,
'Section5Class_add': <MWp instance at 011FA4E0>,
'Section5Class_addForm': <MWp instance at 011FA8B0>,
'Section5Class_factory': <MWp instance at 011FA4C0>,
'StuffSection2': <MWp instance at 011F9E30>,
'StuffSection2_add': <MWp instance at 011C57C0>,
'StuffSection2_addForm': <MWp instance at 011F9140>,
'StuffSection2_factory': <MWp instance at 011FA3C0>,
'StuffSectionClass': <MWp instance at 011FAF60>,
'StuffSectionClass_add': <MWp instance at 011FABE0>,
'StuffSectionClass_addForm': <MWp instance at 011C71C0>,
'StuffSectionClass_factory': <MWp instance at 011FA530>,
'__ac_permissions__': (('Add Section objects', (), ('Manager',)),),
'__doc__': 'StuffClass',
'__module__': '*gK7YKWqDX9Dn00XJH1ZJPQ==',
'_p_changed': None,
'_p_jar': <ZODB.Connection.Connection instance at 11ec6a0>,
'_p_oid': '\000\000\000\000\000\000\014Q',
'_p_serial': '\0032\254\306`\335\272\346',
'_zclass_method_meta_types': ({'action': 'StuffSectionClass_factory',
'name': 'Section object',
'permission': 'Add Section objects',
'product': 'methods'},
{'action': 'Section5Class_factory',
'name': 'Section5 object',
'permission': 'Add Section objects',
'product': 'methods'},
{'action': 'StuffSection2_factory',
'name': 'Section2 object',
'permission': 'Add Section objects',
'product': 'methods'},
{'action': 'section3Class_factory',
'name': 'Section3 object',
'permission': 'Add Section objects',
'product': 'methods'},
{'action': 'myClass_factory',
'name': 'myClass object',
'permission': 'Add myClass objects',
'product': 'methods'}),
'body': '',
'heading': '',
'icon': '',
'isPrincipiaFolderish': 'Y',
'manage_options': ({'action': 'propertysheets/Properties/manage',
'label': 'Properties'},
{'label': 'Contents', 'action': 'manage_main'}),
'meta_type': 'Stuff Site object',
'meta_types': (),
'myClass': <MWp instance at 011F9EA0>,
'myClass_add': <MWp instance at 011C8210>,
'myClass_addForm': <MWp instance at 011F9430>,
'myClass_factory': <MWp instance at 011FA740>,
'propertysheets': <StuffClass_PropertySheetsClass instance at 011FAA50>,
'section3Class': <MWp instance at 011B2960>,
'section3Class_add': <MWp instance at 011FAE60>,
'section3Class_addForm': <MWp instance at 011F8230>,
'section3Class_factory': <MWp instance at 011F92C0>}
### delving even deeper
>>> for b in app.StuffMag.__class__.__bases__:
... pprint(b.__dict__)
{'__class_init__': <function __class_init__ at a05bf0>,
'__module__': 'ZClasses.ZClass'}
{'__module__': 'ZClasses.ObjectManager',
'__roles__': <PermissionRole instance at 00986D50>,
'_zclass_method_meta_types': (),
'all_meta_types': <function all_meta_types at a6ca90>,
'manage_FTPlist__roles__': <PermissionRole instance at 009D7F30>,
'manage_FTPstat__roles__': <PermissionRole instance at 009D7F30>,
'manage_delObjects__roles__': <PermissionRole instance at 009D4FF0>,
'manage_exportObject__roles__': <PermissionRole instance at 00A4D890>,
'manage_importExportForm__roles__': <PermissionRole instance at 00A4D890>,
'manage_importObject__roles__': <PermissionRole instance at 00A4D890>,
'manage_main__roles__': <PermissionRole instance at 00980DF0>,
'manage_menu__roles__': <PermissionRole instance at 00980DF0>,
'meta_types': (),
'objectIds__roles__': <PermissionRole instance at 00986D50>,
'objectItems__roles__': <PermissionRole instance at 00986D50>,
'objectValues__roles__': <PermissionRole instance at 00986D50>}
{'__ac_permissions__': (('View', ()),),
'__doc__': 'Mix-in class combining the most common set of basic mix-ins\012
',
'__module__': 'OFS.SimpleItem',
'manage_options': ({'label': 'Security', 'action': 'manage_access'},)}
### Hmm, why does a propertysheet have no attributes or methods?
>>> pprint(app.StuffMag.propertysheets.__dict__)
{}
### Lets look a bit deeper again
>>> pprint(app.StuffMag.propertysheets.__class__.__dict__)
{'Properties': <ZInstanceSheet instance at 011FA320>,
'__doc__': 'StuffClass Property Sheets',
'__module__': None,
'__propset_attrs__': ('Properties',),
'_p_changed': None,
'_p_jar': <ZODB.Connection.Connection instance at 11ec6a0>,
'_p_oid': '\000\000\000\000\000\000\014P',
'_p_serial': '\0032\2441d!Pn',
'icon': ''}
### Properties looks like what we want
>>> app.StuffMag.propertysheets.Properties
<ZInstanceSheet instance at 011FA320>
### Lets take a look inside
>>> pprint(app.StuffMag.propertysheets.Properties.__dict__)
{'_md': {}, '_base': <ZCommonSheet instance at 012283F0>, 'id':
'Properties'}
### Hmm, have to look at the class again
>>> pprint(app.StuffMag.propertysheets.Properties.__class__.__dict__)
{'_Access_contents_information_Permission': '_View_Permission',
'_Manage_properties_Permission': '_Manage_properties_Permission',
'__ac_permissions__': (('Manage properties',
('manage_addProperty',
'manage_editProperties',
'manage_delProperties',
'manage_changeProperties',
'manage')),
('Access contents information',
('hasProperty',
'propertyIds',
'propertyValues',
'propertyItems',
''))),
'__doc__': 'Waaa this is too hard',
'__module__': 'ZClasses.Property',
'__roles__': <PermissionRole instance at 009E8EF0>,
'hasProperty__roles__': <PermissionRole instance at 009E8EF0>,
'manage__roles__': <PermissionRole instance at 009E7820>,
'manage_addProperty__roles__': <PermissionRole instance at 009E7820>,
'manage_changeProperties__roles__': <PermissionRole instance at 009E7820>,
'manage_delProperties__roles__': <PermissionRole instance at 009E7820>,
'manage_editProperties__roles__': <PermissionRole instance at 009E7820>,
'propertyIds__roles__': <PermissionRole instance at 009E8EF0>,
'propertyItems__roles__': <PermissionRole instance at 009E8EF0>,
'propertyValues__roles__': <PermissionRole instance at 009E8EF0>,
'v_self': <function v_self at a021a0>}
### Hmm, lots of stuff, but I cant see what Im looking for, I'll guess that
acquisition plays a role here
### propertyItems under the permission 'Access contents information' looks
awfully good
>>> app.StuffMag.propertysheets.Properties.propertyItems
<Python Method object at b50970>
### Wahooo - we have arrived,
>>> app.StuffMag.propertysheets.Properties.propertyItems()
[('heading', 'HI THERE!!!'), ('body', '')]
Hmm, it only took an hour or so to figure this out.
> -----Original Message-----
> From: zope-dev-admin@zope.org [mailto:zope-dev-admin@zope.org]On Behalf
> Of Michel Pelletier
> Sent: Monday, February 28, 2000 2:24 PM
> To: pam@digicool.com; zdp@zope.org
> Cc: zope-dev@zope.org; support@digicool.com
> Subject: [Zope-dev] The Debugger Is Your Friend
>
>
>
> And this document tells you why! Last week James W. Howe expressed pain
> in now knowing how to debug Zope, or even if it is possible. Well, it
> is, there is even a Zope debugger! But, alas, until now it was
> undocumented.
>
> So here it is folks, 'The Debugger Is Your Friend'. This is a FIRST
> DRAFT so please read it and send your comments and criticisms to me.
>
> The Debugger Is Your Friend
>
> Jim Fulton is fond of the expression 'The debugger is your friend'.
> And although debugging Zope can sometimes be a bit painful, once you
> are used to the rythum the debugger truly is your friend, and will
> save you years off your life.
>
> Zope has been designed to work in an integrated way with the Python
> debugger (pdb). In order to really be able to use this document,
> you must know how to use pdb. pdb is pretty simple as command line
> debuggers go, and anyone familiar with other popular command line
> debuggers (like gdb) will feel right at home in pdb. If you are
> more familiar with the graphical, bells-and-whistles type debugger,
> I suggest you read the pdb documentation in the standard Python
> module documentation on the "Python website":http://www.python.org/.
>
> For the purposes of this document, I will refer to the debugger as
> the 'Zope debugger', even though most of it is actually the Python
> debugger. This is not to dis the python people, it's just a handy
> convention. Whenever I refer to just the debugger that is not in
> the context of debugging Zope, I will say 'pdb'.
>
> To debug Zope, you must first shut down the Zope process. It is not
> possible to debug Zope and run it at the same time, as the debugger
> stops the entire process dead in its tracks (Note: If you are using
> ZEO, it is possible to run Zope and debug it simultaneously). Also,
> starting up the debugger will by default start Zope in single
> threaded mode. It is not possible to run the debugger and Zope in
> multi-threaded mode at the same time. This is normally not an issue
> unless you are doing some pretty complex low level stuff, which you
> shouldn't be messing with anyway if you have to read this, right?
>
> For most Zope developer's purposes, the debugger is needed to debug
> some sort of application level programming error. A common scenario
> is when developing a new third party Product for Zope. Products
> extend Zope's functionality but they also present the same kind of
> debugging problems that are commonly found in any programming
> environment. It is useful to have an existing debugging
> infrastructure to help you jump immediatly to your new object and
> debug it and play with it directly in pdb. The Zope debugger lets
> you do this.
>
> It is also useful to actually debug Zope itself. For example, if
> you discover some sort of obscure bug in Zope's object request
> broker (ZPublisher) it would be useful to tell Zope to stop at a
> certain point in ZPublisher and bring up the debugger. The Zope
> debugger also lets you do this.
>
> In reality, the 'Zope' part of the Zope debugger is actually just a
> handy way to start up Zope with some pre-configured break points and
> to tell the debugger where in Zope you want to start debugging.
> I'll stop talking now and give some examples.
>
> Remember, for this example to work, you MUST shut down Zope.
> Debugging Zope starts in Zope's 'lib/python' directory. 'lib' is a
> sub-directory of Zope's top level directory. Go to the 'lib/python'
> directory and fire up Python 1.5.2::
>
> Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on
> win32
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import Zope, ZPublisher
> >>>
>
> Here we have run the python interpreter (which is where using the
> debugger takes place) and imported two modules, 'Zope' and
> 'ZPublisher'. If Python complains about an ImportError and not
> being able to find either module, then you are probably in the wrong
> directory, or you have not compiled Zope properly.
>
> The Zope module is the main Zope application. Now, in most cases,
> the term 'application' when used in Zope means some sort of
> application developed on top of Zope, but in this case, we mean Zope
> itself is the application. The ZPublisher module is the Zope 'ORB'
> (object request broker) and is needed to run the debugger.
>
> If trying to import these modules results in a 'cannot lock
> database' error, then another Zope process is using your database.
> Shut that Zope down if you want to debug it. If you get error about
> 'cannot open file such-and-such' then you do not have proper
> permission to read or write one of Zope's files. Get the right
> permission and try again.
>
> At this point, if python did not complain, then all is well and you
> are on your way to debugging Zope. First, let's try something
> really neat. Zope is, as you should know, completely protocol
> agnostic. You do not need a webserver to actually call and use
> Zope, you can do it right here from this python prompt! To
> illustrate this, let's call your Zope site and ask it for the very
> root level URL. If your Zope website's URL is
> 'http://www.mysite.org/', then the very root level URL is '' (or
> '/'). Calling this object with the debugger is as easy as, and just
> like *calling it through the web*. The debugger will set up all
> kinds of fake environment to make Zope not know the difference
> between you calling it here with the debugger and accessing your top
> level root object::
>
> >>> ZPublisher.Zope('')
> Status: 200 OK
> X-Powered-By: Zope (www.zope.org), Python (www.python.org)
> Content-Length: 1238
> Content-Type: text/html
>
> <HTML><HEAD><TITLE>Zope</TITLE>
>
> ... blah blah...
>
> </BODY></HTML>
> >>>
>
> Wow, wasn't that cool? If you look closely, you will see that the
> content returned is *exactly* that which is returned when you call
> your root level object through HTTP, including all the HTTP headers.
>
> So now let's suppose that you have an object in your Zope's root
> level folder called 'aFoo'. aFoo is an instance of your Foo class
> You can call your aFoo instance with the debugger like so::
>
> >>> ZPublisher.Zope('aFoo')
>
> (or you could say '/aFoo' or '/aFoo/' etc..., the rules are the same
> as HTTP). This is a handy way to test out your objects without ever
> leaving python.
>
> Now, let's get to actually debugging your object. Debugging,
> typically, involves running your 'program' up to a point where you
> think it's failing, and then inspecting the state of your variables
> and objects. The easy part is the actual inspection, the hard part
> is getting your program to stop at the right point. For example, if
> you suspect your logic is failing right at the one hundredth
> iteration in a large for loop, then you need to cleverly construct a
> breakpoint to stop your program right before this iteration.
>
> The actual techniques of doing things like this, conditional
> breakpoints and other advanced bebugging, is beyond the scope of
> this document. Many debugger come with documentation, but few come
> with an explanation of the 'Zen' of debugging. Your best bet is to
> play with the debugger as much as possible, and to even use it to
> help you activly engineer your software.
>
> So, for the sake or example, let's say that your 'aFoo' object is
> defined in a Zope Product called 'FooProduct', and is located in the
> 'lib/python/Products/FooProduct' directory. The class that defines
> the 'aFoo' instance is also called 'aFoo', and is defined in the
> 'aFoo.py' module in your Product. Therefore, from Zope's
> perspective, your aFoo's classes fully qualified name is
> 'Products.FooProduct.aFoo.aFoo'. All Zope objects have this
> kind of fully qualified name. For example, the ZCatalog class can
> be found in 'Products.ZCatalog.ZCatalog.ZCatalog (The redundancy is
> because the Product, Module, and class are all named 'ZCatalog').
>
> Now, your 'aFoo' class defines a method called 'fooMethod' that is
> definatly not doing the right thing for you. Here is a possible
> definition of your class and method::
>
> class aFoo:
>
> def fooMethod(self, N):
> """ return the sum from 1 to N """
> sum = 0
> for x in range(N):
> sum = sum + x
> return sum
>
>
> This method can be called with N=5 through the web by visiting
> 'http://www.mysite.org/aFoo/fooMethod?N=5'. This will make Zope
> return the value '15', renderered into an HTML document. There's
> actually nothing wrong with this method to debug, but for the sake
> of argument let's suppose that returning the sumation of N is not
> what we wanted, and therefore we want to debug this method.
>
> Well then, let's fire up the debugger! This is done in a very
> similar way to how we called Zope through the python interpreter
> before, except that we introduce one new argument to the call to
> 'Zope'::
>
> >>> ZPublisher.Zope('/aFoo/fooMethod?N=5', d=1)
> * Type "s<cr>c<cr>" to jump to beginning of real publishing process.
> * Then type c<cr> to jump to the beginning of the URL traversal
> algorithm.
> * Then type c<cr> to jump to published object call.
> > <string>(0)?()
> pdb>
>
>
> Here, we are calling Zope from the interpreter, just like before,
> but there are two differences. First, we are specificaly calling
> our method ('fooMethod') with an argument, and second, we have
> provided a new argument to the Zope call, 'd=1'. The 'd' argument,
> when true, causes Zope to fire up in the python debugger, pdb.
> Notice how the python prompt changed from '>>>' to 'pdb>'. This
> indicates that we are in the debugger.
>
> When you first fire up the debugger, Zope gives you a helpfull
> message that tells you how to get to your object. To understand
> this message, it's useful to know how we have set Zope up to be
> debugged. When Zope fires up in debugger mode, there are three
> breakpoints set for you automatically (if you don't know what a
> breakpoint is, you need to read the python debugger documentation).
> The first breakpoint stops the program at the point that ZPublisher
> (the Zope ORB) tries to publish the application module (in this
> case, the application module is 'Zope'). The second breakpoint
> stops the program right before ZPublisher tries to traverse down the
> provided URL path (in this case, '/aFoo/fooMethod'). The third
> breakpoint will stop the program right before ZPublisher calls the
> object it finds that matches the URL path (in this case, the 'aFoo'
> object).
>
> So, the little blurb that comes up and tells you some keys to press
> is telling you these things in a terse way. Hitting 's' will 's'tep
> you into the debugger, and hitting 'c' will 'c'ontinue the execution
> of the program until it hits a breakpoint.
>
> Note however that none of these breakpoints will stop the program at
> 'fooMethod'. To stop the debugger right there, you need to tell the
> debugger to set a new breakpoint. This is done quite easily.
> First, you must import your Python module (in this case, a Zope
> Product called 'aFoo', note the above discussion on 'fully
> qualified' names).
>
> pdb> import Products
> pdb> b Products.FooProduct.aFoo.aFoo.fooMethod
> Breakpoint 5 at C:\Program
> Files\WebSite\lib\python\Products\FooProduct\aFoo.py:42
> pdb>
>
> First, we import 'Products'. Since your module is a Zope Products,
> it can be found in the Products Module. Next, we set a new
> breakpoint with the 'b'reak debugger command (pdb allows you to use
> single letter commands, but you could have also used the entire word
> 'break'). The breakpoint we set is
> 'Products.FooProduct.aFoo.aFoo.fooMethod'. After setting this
> breakpoint, the debugger will respond that it found the method in
> question in a certain file, on a certain line (in this case, the
> ficticious line 42) and return you to the debugger.
>
> Whew! Lots of stuff to take in there in a few short paragraphs, but
> you are now on your way to debugging your method with style. Now,
> we want to get to our 'fooMethod' so we can start debugging it.
> But along the way, we must first 'c'ontinue through the various
> breakpoints that Zope has set for us. Although this may seem like a
> bit of a burden, it's actually quite good to get a feel for how Zope
> works internally by getting down the rythum that Zope uses to
> publish your object. In these next examples, my inline comments
> will begin with '#". Obviously, you won't see these comments when
> you are debugging. So let's debug::
>
> pdb> s
> # 's'tep into the actual debuging
>
> > <string>(1)?()
> # this is pdb's response to being steped into, ignore it
>
> pdb> c
> # now, let's 'c'ontinue onto the next breakpoint
>
> > C:\Program
> Files\WebSite\lib\python\ZPublisher\Publish.py(112)publish()
> -> def publish(request, module_name, after_list, debug=0,
>
> # pdb has stoped at the first breakpoint, which is the point where
> # ZPubisher tries to publish the application module.
>
> pdb> c
> # continuing onto the next breakpoint we get...
>
> > C:\Program
> Files\WebSite\lib\python\ZPublisher\Publish.py(101)call_object()
> -> def call_object(object, args, request):
>
> # Here, ZPublisher (which is now publishing the application) has
> # found your object and is about to call it. Let's jump out here
> # and discuss this point...
>
> # 'Calling' your object consists of applying the arguments supplied
> by
> # ZPublisher against the object. Here, you can see how ZPublisher
> is
> # passing three arguments into this process. The first argument is
> # 'object' and is the actual object you want to call. This can be
> # verified by 'p'rinting the object::
>
> pdb> p object
> <aFoo instance at 00AFE410>
>
> # Neat! So here you can inspect your object (with the 'p'rint
> # command) and even play with it a bit, but we're not there yet!
> #
> # The next argument is 'args'. This is a tuple of arguments that
> # ZPublisher will apply against your object call.
> #
> # The final argument is 'request'. This is the 'request object' and
> # will eventually be transformed in to the DTML usable object
> 'REQUEST'.
>
> pdb> c
>
> # Now let's continue, which breakpoint is next? Yours!
>
> > C:\Program
> Files\WebSite\lib\python\Products\FooProduct\aFoo.py(42)fooMethod()
> -> def fooMethod(self, N)
>
> # and now we are here, at YOUR method. How can we be sure, well,
> # let's tell the debugger to show us where we are in the code:
>
>
> pdb> l
> 41 def fooMethod(self, N):
> 42 """ return the sum from 1 to N """
> 43-> sum = 0
> 44 for x in range(N):
> 45 sum = sum + x
> 46 return sum
>
>
> And that's it, really. From here, with a little knowledge of the
> python debugger, you should be able to do any kind of debugging task
> that is needed.
>
> _______________________________________________
> Zope-Dev maillist - Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope )
>