I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient) In a nutshell, it patches the rendering guts of DTMLMethod, ZSQL, etc. and the ZPublisher so that we can get timing marks for requests as the calls are made. The patching is quite simple (once I got the hang of it ;) and reversible, and means that there's no changes required to the Zope core to enable the testing (which our initial versions did). It does pose a question though: would it be better to have support for this stuff in the core, or is it OK for a product like this to dynamically patch the call methods when it needs to? On one hand, the performance hit when dynamically patching the methods is zero when the product is not "active", but it does mean diddling with methods that really probably should be left alone. On the other hand, having changes to the core code to test for profiling being enabled introduces a small performance hit even when profiling is not activated. Any comments? Richard
On 1/2/02 1:13 am, "Richard Jones" <rjones@ekit-inc.com> wrote:
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
[snip]
Any comments?
I've been using it for *5* minutes and have found a method that was taking 95%! of one of our most often used scripts and another python script that was effectively stalling the server....gobsmacked. Huge kudos and thanks to Richard!! :) Tone. -- Dr Tony McDonald, Assistant Director, FMCC, http://www.fmcc.org.uk/ The Medical School, Newcastle University Tel: +44 191 243 6140 A Zope list for UK HE/FE http://www.fmcc.org.uk/mailman/listinfo/zope
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
This is absolutely excellent, well done and thanks :) I added various FSObjects to the profiled modules list and have been looking at my CMF systems - it's really instructive.
It does pose a question though: would it be better to have support for this stuff in the core, or is it OK for a product like this to dynamically patch the call methods when it needs to?
I see nothing wrong with it being a monkeypatch product. On the other hand, it is a very useful tool, and as such could benefit from being in the core simply so that it always gets maintained with the core.
On one hand, the performance hit when dynamically patching the methods is zero when the product is not "active", but it does mean diddling with methods that really probably should be left alone. On the other hand, having changes to the core code to test for profiling being enabled introduces a small performance hit even when profiling is not activated.
You could do this check only if zope is being run in debug mode; then there would be zero perormance hit in production sites.
Any comments?
I think the reporting could do with some usability improvements. It's good now, but takes a while to navigate round. - All the reports could benefit from being sortable by column. - the trace for a complete template could be slightly clearer. The colour coding for the start and end of a module could have a clear legend, for example. - there needs to be some solution for really long URLs, which require lots of horizontal scrolling - just displaying the final parts of the path, with the rest of the url hidden somehow? I can't think of exactly what else I would do to improve it right now, though. And that's the kind of change I never get round to bothering with, personally... Sorry these aren't really constructive ideas, but you did ask for feedback :-) seb
On Fri, 1 Feb 2002 21:53, seb bacon wrote:
I wrote:
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
This is absolutely excellent, well done and thanks :) I added various FSObjects to the profiled modules list and have been looking at my CMF systems - it's really instructive.
Please send me the config entries for them. Also, if anyone has used ZPT with it, let me know if my guess of the config for them is correct :)
On one hand, the performance hit when dynamically patching the methods is zero when the product is not "active", but it does mean diddling with methods that really probably should be left alone. On the other hand, having changes to the core code to test for profiling being enabled introduces a small performance hit even when profiling is not activated.
You could do this check only if zope is being run in debug mode; then there would be zero perormance hit in production sites.
No, there's still the performance check for "am I in debug mode?" which is equivalent to "am I in profiling mode?". Sure, it's small, but it's there.
Any comments?
I think the reporting could do with some usability improvements. It's good now, but takes a while to navigate round.
Yep, I know :)
- All the reports could benefit from being sortable by column.
In the works already :)
- the trace for a complete template could be slightly clearer. The colour coding for the start and end of a module could have a clear legend, for example.
I'll put a legend in. Is the colour coding what you meant by "clearer" or is there something else?
- there needs to be some solution for really long URLs, which require lots of horizontal scrolling - just displaying the final parts of the path, with the rest of the url hidden somehow?
Yep, that's a good idea.
I can't think of exactly what else I would do to improve it right now
I've already got some more code in place that I'm testing right now which does aggregation of matching result profiles. That is, if a request results in the same call profile, the timings are aggregated into min/average/max. Really neat :) Thanks for the feedback! Richard
On Sun, 2002-02-03 at 21:58, Richard Jones wrote:
On Fri, 1 Feb 2002 21:53, seb bacon wrote:
I wrote:
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
This is absolutely excellent, well done and thanks :) I added various FSObjects to the profiled modules list and have been looking at my CMF systems - it's really instructive.
Please send me the config entries for them. Also, if anyone has used ZPT with it, let me know if my guess of the config for them is correct :)
in profileable_modules: 'FSPythonScript': Profileable('Products.CMFCore.FSPythonScript', 'FSPythonScript', '__call__'), 'FSDTMLMethod': Profileable('Products.CMFCore.FSDTMLMethod', 'FSDTMLMethod', '__call__'), 'FSPageTemplate': Profileable('Products.CMFCore.FSPageTemplate', 'FSPageTemplate', '__call__'),
You could do this check only if zope is being run in debug mode; then there would be zero perormance hit in production sites.
No, there's still the performance check for "am I in debug mode?" which is equivalent to "am I in profiling mode?". Sure, it's small, but it's there.
- the trace for a complete template could be slightly clearer. The colour coding for the start and end of a module could have a clear legend, for example.
I'll put a legend in. Is the colour coding what you meant by "clearer" or is there something else?
Yes, that's what I meant. Actually, another thing just occurred to me: the meaning of the "..." ellipsis is not clear - I think you need a different symbol to emphasise the hierarchy (perhaps a vertical bar?) Cheers, seb
On Mon, 4 Feb 2002 21:42, seb bacon wrote:
On Sun, 2002-02-03 at 21:58, Richard Jones wrote:
On Fri, 1 Feb 2002 21:53, seb bacon wrote:
I wrote:
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
This is absolutely excellent, well done and thanks :) I added various FSObjects to the profiled modules list and have been looking at my CMF systems - it's really instructive.
Please send me the config entries for them. Also, if anyone has used ZPT with it, let me know if my guess of the config for them is correct :)
in profileable_modules:
'FSPythonScript': Profileable('Products.CMFCore.FSPythonScript', 'FSPythonScript', '__call__'), 'FSDTMLMethod': Profileable('Products.CMFCore.FSDTMLMethod', 'FSDTMLMethod', '__call__'), 'FSPageTemplate': Profileable('Products.CMFCore.FSPageTemplate', 'FSPageTemplate', '__call__'),
Thanks!
- the trace for a complete template could be slightly clearer. The colour coding for the start and end of a module could have a clear legend, for example.
I'll put a legend in. Is the colour coding what you meant by "clearer" or is there something else?
Yes, that's what I meant.
OK, it's done. Will be in the next release (probably today :)
Actually, another thing just occurred to me: the meaning of the "..." ellipsis is not clear - I think you need a different symbol to emphasise the hierarchy (perhaps a vertical bar?)
OK, I'll look into it. The '...' is definitely a placeholder :) Richard
Doesnt seem to like being refreshed and some event items arent in the time dict, I noticed these were items that I had got in a ram cache. So i just stuck an if at line 496. Other than that its a great product. Most important tool since refresh if you ask me... ----- Original Message ----- From: "Richard Jones" <rjones@ekit-inc.com> To: "zope-dev" <zope-dev@zope.org> Sent: Thursday, January 31, 2002 5:13 PM Subject: [Zope-dev] Call Profiler
I've just announced our Call Profiler product on zope.org and the zope announce list (waiting for people in different timezones to authorise them :) ( http://www.zope.org/Members/richard/CallProfiler/ for the impatient)
In a nutshell, it patches the rendering guts of DTMLMethod, ZSQL, etc. and the ZPublisher so that we can get timing marks for requests as the calls are made. The patching is quite simple (once I got the hang of it ;) and reversible, and means that there's no changes required to the Zope core to enable the testing (which our initial versions did).
It does pose a question though: would it be better to have support for this stuff in the core, or is it OK for a product like this to dynamically patch the call methods when it needs to?
On one hand, the performance hit when dynamically patching the methods is zero when the product is not "active", but it does mean diddling with methods that really probably should be left alone. On the other hand, having changes to the core code to test for profiling being enabled introduces a small performance hit even when profiling is not activated.
Any comments?
Richard
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
On Sat, 2 Feb 2002 02:49, Zopista wrote:
Doesnt seem to like being refreshed and some event items arent in the time dict, I noticed these were items that I had got in a ram cache. So i just stuck an if at line 496. Other than that its a great product. Most important tool since refresh if you ask me...
Yeah, as the README states - the profiling data is memory-only - I'm going to look into a post-publish hook that stuffs the result from a single run into the ZODB. Richard
participants (4)
-
Richard Jones -
seb bacon -
Tony McDonald -
Zopista