RE: [Zope] Options for handling concurrency?
My methods are more complex than simple ZSQL lookups. I need to do a tree traversal through a database to get the relevant items to display. So I need to do nested SQL queries.The first Zope attempt was ;
<dtml-var standard_html_header> <dtml-if child><dtml-var show_html> <dtml-in expr="Actors.SQL.get_children(parent=child)"><dtml-var show_html> <dtml-in expr="Actors.SQL.get_children(parent=child)"><dtml-var show_html> <dtml-in expr="Actors.SQL.get_children(parent=child)"><dtml-var show_html> <dtml-in expr="Actors.SQL.get_children(parent=child)"><dtml-var show_html></dtml-in get_children L4></dtml-in get_children L3></dtml-in get_children L2></dtml-in get_children L1> <dtml-else> <form action='cascade2' method=post> child <input type=text name=child size=10> </form> </dtml-if> <dtml-var standard_html_footer>
That is, it's a nested loop of ZSQL methods and show_html is an external method that displays HTML dependant on the contents of a DTML document in the ZODB (this is *so* cool - I can change the look and feel of 'elements' with triviality...) It took about 27 seconds to render a (120k) document.
Using a full External Method solution, with calls to DBH.query (and a one-time lookup for all the DTML 'templates' in the ZODB) it took about 2 seconds (it's not completely debugged though! :).
So, I'm *not* calling ZSQL methods from an External method, I'm using direct connections to the database using Python libraries. Obviously, I lose a lot of Zope goodies here, but I really need that speedup...
So, if I understand you correctly, your EM's call query on the DA object itself, not using the ZSQL Methods. Then you have no cache whatsoever on those queries. Is there no way you can use a ZSQL Method here anyway? Just a self.nameOfZSQLMethod(arguments) might speed things up here. Also, I don't know how your recursive query will work within the one thread of a HTTP request. What python libraries do you use? -- Martijn Pieters, Software Engineer | Digital Creations http://www.digicool.com | Creators of Zope http://www.zope.org | mailto:mj@digicool.com ICQ: 4532236 | PGP: http://wwwkeys.nl.pgp.net:11371/pks/lookup?op=get&search=0xA8A32149 -------------------------------------------
At 11:29 am -0500 29/2/00, Martijn Pieters wrote:
So, I'm *not* calling ZSQL methods from an External method, I'm using direct connections to the database using Python libraries. Obviously, I lose a lot of Zope goodies here, but I really need that speedup...
So, if I understand you correctly, your EM's call query on the DA object itself, not using the ZSQL Methods. Then you have no cache whatsoever on those queries. Is there no way you can use a ZSQL Method here anyway? Just a self.nameOfZSQLMethod(arguments) might speed things up here.
Apologies for the late reply - flu :( Yes. I create a DBH in an EM and fire away. All I get from the Zope system is a request of the form http://supercascade?root_id=240551 and I start to do my queries. They are nested in the EM to 9 levels deep (but normally the first 4 levels are enough to get all the nodes of the tree with the relevant root_id). Getting the list of nodes is quite fast, about 0.8 seconds. Then I have to render them. I've got that down by a factor of ten (since this thread has started in actual fact!) I've also found out just how fast a cached ZSQL method can be - one DMTL method that used query took 3.2 seconds on the first render, and 0.06 seconds on subsequent renders. Obviously some more work needs to be done in this area and I'm looking into that. I'm also told that our MySQL server itself does some caching ... looking into that.
Also, I don't know how your recursive query will work within the one thread of a HTTP request.
Ah, I hadn't thought of that...my query isn't recursive however.
What python libraries do you use?
MySQL (MySQLmodule-1.4), DocumentTemplate and from Acquisition,Implicit in the main. thanks for the thoughts Martijn, Tone. ------ Dr Tony McDonald, FMCC, Networked Learning Environments Project http://nle.ncl.ac.uk/ The Medical School, Newcastle University Tel: +44 191 222 5888 Fingerprint: 3450 876D FA41 B926 D3DD F8C3 F2D0 C3B9 8B38 18A2
participants (2)
-
Martijn Pieters -
Tony McDonald