From: Tony McDonald [mailto:tony.mcdonald@ncl.ac.uk]
At 9:43 am +0100 27/2/00, Martijn Pieters wrote:
- Have a look at /Control_Panel/manage_debug. It'll show you
reference
counts and currenlty open connections, which can be very helpful in targeting hotspots.
Aye carumba! - what a lot of information. I'm going to start delving around there and see what happens. Is there any additional documentation available on what the info. means? - I've looked in the ApplicationManager.py code is a wee bit over my head.
It shows you reference counts, and how they change over time. Reference counts are how many references there are to classes and methods. This generally lets you see how many instances there are of those classes. It allows you to see where the hotspots might be.
As an aside (to the list), looking around the ApplicationManager.py reveals something called manage_profile. It looks like the main Zope functions are being profiled here (ie not at the object level). It's interesting, but it does seem to be a resource hog...
But so is something in your app. You might want to run it and see what analysis you get. For more documentation of the data the profiler returns, see: http://www.python.org/doc/lib/profile.html
- Also, what DBMS are you using? If you are using a level 1 or 2 DA (see http://www.zope.org/Members/petrilli/DARoadmap) then you can do only one DB query at a time. It might be time to choose a different DB, or if you are using the old MySQL DA, to upgrade.
It's MySQL, but using the latest DA - I'm pretty sure I'm doing more than one query at a time as I set off a ZSQL query that takes 9 seconds from the MySQL command line, and then started to use aspects of the site that I *know* use MySQL lookups. BTW, the 9 second SQL query isn't used in the site! :)
This only shows that MySQL is multithreaded, not that the DA is. Run the expensive, 9 second query in a ZSQL Method. Then try and access another ZSQL Method on the same Zope server. If it is multithreaded, you should get results about as fast as when the heavy one isn't running. If it is the old MySQL DA, you'll have to wait until the 9 sec query is finished.
- If you are using ZSQL a lot, do you use caching? Even a cache expiry of 1 second would help a heavily hit site tremendously, noone will notice the data returned being at most 1 second out of date.But all hits with equal parameters on the same thread in Zope within that second need only one DB query.
I do use caching Martijn, but I must admit that it's been a hit and miss affair. I'm trying to do as many of my SQL queries in external methods (ironically for the performance), so I may well be losing a lot of the caching performance breakthrough there .... hmmm. I wonder if I could do something like...
This shouldn't make any difference. Only in doing excessive loops in loops and the like, would an External Method be any use. -- Martijn Pieters, Software Engineer | Digital Creations http://www.digicool.com | Creators of Zope http://www.zope.org | mailto:mj@digicool.com ICQ: 4532236 | PGP: http://wwwkeys.nl.pgp.net:11371/pks/lookup?op=get&search=0xA8A32149 -------------------------------------------