Time-consuming External method blocks Zope execution
Hi all I have an external Python method that can take a long time to complete. This method does some conversions of files on the Zope server into another format on request of a client. My problem is that the whole Zope installation comes to a standstill while this method executes. Nobody else can get any response from the site until the external method have returned. I know that I can make my long process run in a thread and return immediately, but how do I notify the client browser when the thread have finally finished. Is the only way to write a self-reloading page as response for the client so that his browser checks every now and again if the process is finished eg. 1) Client submits form with files to be converted 2) Client receives a please wait page with JavaScript (or whatever) that will check for process completion in 10 sec. 3) Client browser aoutomatically calls a checkIfProcessIsFinished page - gets page at 2) if it is not else gets page 4) 4) If the process was completed he gets a results page I did make sure that my Zope does not run with the -t 0 or -D switches (it does the same running as a service too) Thanks Etienne
On Wed, 13 Jun 2001, Etienne Labuschagne wrote:
I know that I can make my long process run in a thread and return immediately, but how do I notify the client browser when the thread have finally finished. Is the only way to write a self-reloading page as response for the client so that his browser checks every now and again if the process is finished
There's a solution for that, I've used it in a CGI script to solve a browser (or Apache) timeout problem when launching long SQL queries, but have not implemented it in Zope, so I don't know how this would apply to Zope. you have to return a document which content-type is: multipart/x-mixed-replace;boundary=XXX where XXX is some random string. the first line of you document must contain: --XXX (same XXX as before) then you launch a thread which periodically send a normal CGI document (with headers, and content-type=text/html for example) displaying for example the elapsed time since you've begun to convert files, but this document must end with "\n--%s" % XXX (still same XXX as above) then you start to convert files. During the conversion your thread will periodically send a new page to your browser, so you can display in it as many info as you want: elapsed time, number of converted files, etc... when it's finished you end the thread. then you finish the enclosing CGI document which content type is multipart/x-mixed-replace, terminating it with: "\n--%s--" % XXX (still same XXX) then it's done. I don't know if this would work at all in Zope, but I suppose you can try this way. Anyway I can send to you my python CGI script if you want. bye, Jerome Alet
If anybody has done this in Zope, I would be REALLY interessted in getting a close look at the code ! May be through an HowTo on zope.org ? Thanx ! ----- Original Message ----- From: "Jerome Alet" <alet@unice.fr> To: "Etienne Labuschagne" <etiennel@geospace.co.za> Cc: "Zope user list (E-mail)" <zope@zope.org> Sent: Wednesday, June 13, 2001 9:48 AM Subject: Re: [Zope] Time-consuming External method blocks Zope execution
On Wed, 13 Jun 2001, Etienne Labuschagne wrote:
I know that I can make my long process run in a thread and return immediately, but how do I notify the client browser when the thread have finally finished. Is the only way to write a self-reloading page as response for the client so that his browser checks every now and again if the process is finished
There's a solution for that, I've used it in a CGI script to solve a browser (or Apache) timeout problem when launching long SQL queries, but have not implemented it in Zope, so I don't know how this would apply to Zope.
you have to return a document which content-type is:
multipart/x-mixed-replace;boundary=XXX
where XXX is some random string.
the first line of you document must contain:
--XXX (same XXX as before)
then you launch a thread which periodically send a normal CGI document (with headers, and content-type=text/html for example) displaying for example the elapsed time since you've begun to convert files, but this document must end with "\n--%s" % XXX (still same XXX as above)
then you start to convert files.
During the conversion your thread will periodically send a new page to your browser, so you can display in it as many info as you want: elapsed time, number of converted files, etc...
when it's finished you end the thread.
then you finish the enclosing CGI document which content type is multipart/x-mixed-replace, terminating it with:
"\n--%s--" % XXX (still same XXX)
then it's done.
I don't know if this would work at all in Zope, but I suppose you can try this way.
Anyway I can send to you my python CGI script if you want.
bye,
Jerome Alet
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Wed, 13 Jun 2001 10:00:01 +0200, "Olivier Deckmyn" <odeckmyn.list@teaser.fr> wrote:
If anybody has done this in Zope, I would be REALLY interessted in getting a close look at the code !
You can do execatly the same thing using RESPONSE.write to slowly trickle content back to the client. However that request will block one of Zopes publisher threads for the duration of the request. This may or may not be a problem. Toby Dickenson tdickenson@geminidataloggers.com
Jerome Alet writes:
you have to return a document which content-type is:
multipart/x-mixed-replace;boundary=XXX Very Interesting!
* where did you find this (extension) MIME type documented? * what browsers support this? Dieter
On Thu, 14 Jun 2001, Dieter Maurer wrote:
Jerome Alet writes:
you have to return a document which content-type is:
multipart/x-mixed-replace;boundary=XXX Very Interesting!
* where did you find this (extension) MIME type documented?
It's in O'Reilly's "CGI Programming on the World Wide Web", by Shishir Gundavaram, ISBN: 1-56592-168-2, chapter 11: "Advanced and creative CGI applications". unfortunately the examples are in f...ingly unreadable Perl !
* what browsers support this?
I don't know, I've only tested with Netscape 4.xx but I suppose most do. bye, Jerome Alet
----- Original Message ----- From: "Jerome Alet" <alet@unice.fr> Subject: Re: [Zope] Time-consuming External method blocks Zope execution
On Thu, 14 Jun 2001, Dieter Maurer wrote:
* where did you find this (extension) MIME type documented?
It's in O'Reilly's "CGI Programming on the World Wide Web", by Shishir Gundavaram, ISBN: 1-56592-168-2, chapter 11: "Advanced and creative CGI applications".
unfortunately the examples are in f...ingly unreadable Perl !
HEY! I like Perl. And Tcl/Tk by the way... /dario - asbestos protection is: ON. [For the Humour Impaired: This broadcast is a "Joke"]
On 15 Jun 2001 09:37:21 +0200, Dario Lopez-Kästen wrote:
----- Original Message ----- From: "Jerome Alet" <alet@unice.fr> Subject: Re: [Zope] Time-consuming External method blocks Zope execution
On Thu, 14 Jun 2001, Dieter Maurer wrote:
* where did you find this (extension) MIME type documented?
It's in O'Reilly's "CGI Programming on the World Wide Web", by Shishir Gundavaram, ISBN: 1-56592-168-2, chapter 11: "Advanced and creative CGI applications".
unfortunately the examples are in f...ingly unreadable Perl !
HEY! I like Perl. And Tcl/Tk by the way...
Well, there's no accounting for taste. ;^)
On Thu, 14 Jun 2001, Dieter Maurer wrote:
Jerome Alet writes:
you have to return a document which content-type is:
multipart/x-mixed-replace;boundary=XXX Very Interesting!
* where did you find this (extension) MIME type documented?
This is called Server Push (to distinguish from Client Pull): http://home.netscape.com/assist/net_sites/pushpull.html
* what browsers support this?
All major browsers from ancient times. Oleg. ---- Oleg Broytmann http://www.zope.org/Members/phd/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Etienne Labuschagne writes:
I have an external Python method that can take a long time to complete. This method does some conversions of files on the Zope server into another format on request of a client. My problem is that the whole Zope installation comes to a standstill while this method executes. This should not happen!
Is your conversion pure Python? Then, there would be something wrong with Python's thread support. It is supposed to switch threads every <x> instructions, where "<x>" can be controlled by the "-i" option to Zope's "z2.py". If it is done in an external C module, then this module may not have been designed (or compiler) for thread support. If it does not release Python's interpreter lock, then the other Python threads will not get a chance to run before this module function returns. Doing the conversion in its own thread would not help you in either case. While it is possible to let operations be executed in a separate threads, it is not easy to do this in a safe way (see other message titled "Zope transactions and connections"). Dieter
participants (8)
-
Bill Anderson -
Dario Lopez-Kästen -
Dieter Maurer -
Etienne Labuschagne -
Jerome Alet -
Oleg Broytmann -
Olivier Deckmyn -
Toby Dickenson