Progress information for long running process
Hi All, Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this? Thanks a lot! -- Eddie.
Eddie Mönch wrote at 2003-4-8 17:13 +0200:
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
A long time ago, I read an announcement for "LocalProc". When I remember right, this product should facilitate handling of long running processes. I never used it, but maybe, it helps you. Dieter
On Tuesday 08 April 2003 11:13 am, Eddie Mönch wrote:
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
My suggestion would be to spawn this as an external process. Then redirect to a progress page with a meta refresh interval set. You will then have to devise some way for the page refresh to get the status from the process. One way might be for the external process to call back to the server (over XML-RPC), and set the progress value every so often (possibly the same interval as the page refresh). The server would store this in the user's session. The progress page would look in the session for the progress value and draw the progress screen appropriately. When it reached 100%, the progress page would redirect to a final "completed" page. To make this work, the original external method would need to create a random key that was passed to both the external process (on the command line) and the progress bar page (via the URL). That would be the key used to store and access the progress data in the session. If you don't care about tying up a Zope thread for 30 minutes, then you could skip spawning the external process, and just have it write its status to the session directly every so often. The progress bar page could work the same. XML-RPC in python is pretty transparent though, so it would not add that much complexity to your code to use a separate process. hth, -Casey
Use streamed strings at every step completion: Example (using a pythonscript): statement 1 statement 2 context.REQUEST.response.write("statement2, done") statement3 context.REQUEST.response.write("statement3, done") Just take into account that when you use this, all what your page will show is what it was printed out by the response object. If you're willing to use html, make sure to include the closing tags within each response string, as much as possible. To perform a redirection at the moment the stream ends (the messages page gets finally rendered), you may use a simple javascript call placed at the bottom (embeded as the last streamed string), because response.redirect(URL) will not work Ausum. ----- Original Message ----- From: "Eddie Mönch" <Moench@Ontoprise.DE> To: <zope@zope.org> Sent: Tuesday, April 08, 2003 10:13 AM Subject: [Zope] Progress information for long running process
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
Thanks a lot!
-- Eddie.
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Eddie Mönch wrote:
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
I'm not sure, but I would at least consider the problem that the connection might time out anywhere in between the client and your server. While this might be covered by some standards, I have seen SOHO proxy solutions breaking at much more ordinary requirements. cheers, oliver
Eddie M?nch wrote:
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
Actually, I would probably try something completely different. I would immediately return something like: 'Your job is being processed. As this job takes a long time to perform, and there is no good reason to tie up you and your browser, I will send you an email when this is done telling you where you may pick up your results.' Then send an email with an URI to the result. Jim Penny
What about writing the process status into a log-file? After a user starts the process, redirect him to that logfile (e.g. HTML-type with a meta-refresh so that it updates) and let him watch the progress. You may want him to return back, then.. well, in the final step, replace the self-refreshing metatag with one that passes the user back to wherever you want. Just a workaround. :-) -----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Jim Penny Sent: Wednesday, April 09, 2003 5:42 PM To: zope@zope.org Subject: Re: [Zope] Progress information for long running process Eddie M?nch wrote:
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
Actually, I would probably try something completely different. I would immediately return something like: 'Your job is being processed. As this job takes a long time to perform, and there is no good reason to tie up you and your browser, I will send you an email when this is done telling you where you may pick up your results.' Then send an email with an URI to the result. Jim Penny _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
hi, i have the following problem: scenario: i have these 3 files in root folder in zope: /index_html /standard_html_header /standard_html_footer and a file 'a' in 'x' folder /x/a 'a' file contains: <dtml-var standard_html_header> hi <dtml-var standard_html_footer> and 'index_html' file contains this: <dtml-var "x.a()"> when i want to view the 'index_html' file i get: Error Type: KeyError Error Value: standard_html_header it seems that 'a' is evaluated in context where the standard_html_* methods are not inherited. how can i make this work? (ie- i need to call methods that are in subfolders of caller object and i need to have their dtml tags interpolated by zope) regards -dan -- dan meszaros <dm@petamem.com>; programmer; PetaMem, s.r.o., Ocelarska 1, 0x4268 Praha 0x9, Ceska republika; http://www. petamem.com
Dan Meszaros wrote at 2003-4-9 18:48 +0200:
.... 'a' file contains: <dtml-var standard_html_header> hi <dtml-var standard_html_footer>
and 'index_html' file contains this: <dtml-var "x.a()">
when i want to view the 'index_html' file i get: Error Type: KeyError Error Value: standard_html_header
Please read "Calling DTML objects" in <http://www.dieter.handshake.de/pyprojects/zope/book/chap3.html> Dieter
On Wed, Apr 09, 2003 at 05:53:52PM +0200, Jan Maska wrote:
What about writing the process status into a log-file? After a user starts the process, redirect him to that logfile (e.g. HTML-type with a meta-refresh so that it updates) and let him watch the progress.
You may want him to return back, then.. well, in the final step, replace the self-refreshing metatag with one that passes the user back to wherever you want.
Just a workaround. :-)
My first reaction was, not pleasant. On second thought, this is an interesting idea. Why? I think that this immediatley frees up the Zope thread, which is crucial. Expanding on the email idea. It is an honest one. You are freeing up resources doing it that way, both computer resources and people resources. I don't like to watch progress bars that take 30 minutes to complete. More importantly, that progress bar is tying up a browser window on the client, probably a zope thread, and probably a bunch of other things I don't know about. This sounds like a batch process. Trying to make it appear to be interactive is probably a waste of everyones time. The user probably cares about the following: 1) knowing that the task was submitted 2) knowing that he will be told when the task is done, and how to get it. and 3) knowing what to do if the task seems to be taking too long, that is, how long too long is, and who to call if no email is forthcoming. [this should be a part of the submission email.] The user probably does not care to know 'I am at step 39 of 100, the renormalization phase'. If it is important to the user to know this kind of information, or the users demand it, then do all the RESPONSE stuff. Otherwise, avoid it. The email idea is one way of avoiding the whole mess. The redirect to logfile is also interesting; it appears to have many of the same benefits as the email idea. The only detriment is that the client has to keep the browser open to see that the job is completed, and maybe even to see the job results. Jim Penny
Eddie M?nch wrote:
Hi All,
Could someone help with this problem: I have a long running process performed with an external method. Long means > 30 min. At the and I'm giving back HTML code to the calling object. What I need now is same sort of progress information on the web page (e.g. Processing step 50 out of 100). Has anyone an idea how to do this?
Actually, I would probably try something completely different. I would immediately return something like:
'Your job is being processed. As this job takes a long time to perform, and there is no good reason to tie up you and your browser, I will send you an email when this is done telling you where you may pick up your results.'
Then send an email with an URI to the result.
Jim Penny
Other alternatives, if you don't have the email of the user and showing logs is not important, are - A javascript which visibly counts down 30 minutes and the goes to take the result - Return a specific link to the user where he can check himself if there's a result - Offer the user to set the above URI as a bookmark (maybe with javascript). - Set a cookie so that you can remind the user that "there's something waiting for you, click _here_" on adequate pages of the site (front page, user homepage, whatever) - any combination of the above It's all a question of how many resources one is willing/able to spend, I'd prefer to avoid involving zope too much just to show the user some ascii animations for 30 minutes. cheers, oliver
Just to chime in another option: If your clients are not techies and want a status bar, it would be quite easy to establish a small status bar as an image, then refresh it via javascript. Zope can respond with an image whose width attribute is proportional to the percent complete. If the task is complete, Zope can respond with an entirely different image, saying 'Done!' Zope could also respond with an error image. This would give you a pretty status bar and a relatively low load on zope. If it were me, I'd probably do this if the task took about 5 minutes. But if it took more like an hour, I'd go with the external email.
Yeah, right. :-) This was just a quickly made-up trick. It obviously *IS* a nonsense to use it on operations that may take ages (as those 30 minutes before you go home :-))) I'd recommend it though, for operations that take from a minute to - let's say ten minutes. I believe everyone's pleased with knowing exactly what's going on behind the Zope walls. :-) Furthermore, it could eventualy report a problem during the operation. If well-thought, it could really supply an almost-interactive interface. For example: your operation consists of two or more iterations. In the first one it gathers needed data and in the second it feeds them to an important repository, sends them by email to your boss's boss or interacts with a remote API. Now, imagine the operation didn't get valid data. That's not a failure of the script, thus it doesn't die(). Using this technique, it can pause the process and require an interaction from you.. " This and that happened. Do you want to: a) Abort, b) Restart the process c) Continue anyway" Which is impossible by the email. But I admit that you'd pobably need this type of interaction is very rarely. :-) BTW, I like the idea of sending an ETA with administration contacts via email. -Mac ================================================= My first reaction was, not pleasant. On second thought, this is an interesting idea. Why? I think that this immediatley frees up the Zope thread, which is crucial. Expanding on the email idea. It is an honest one. You are freeing up resources doing it that way, both computer resources and people resources. I don't like to watch progress bars that take 30 minutes to complete. More importantly, that progress bar is tying up a browser window on the client, probably a zope thread, and probably a bunch of other things I don't know about. This sounds like a batch process. Trying to make it appear to be interactive is probably a waste of everyones time. The user probably cares about the following: 1) knowing that the task was submitted 2) knowing that he will be told when the task is done, and how to get it. and 3) knowing what to do if the task seems to be taking too long, that is, how long too long is, and who to call if no email is forthcoming. [this should be a part of the submission email.] The user probably does not care to know 'I am at step 39 of 100, the renormalization phase'. If it is important to the user to know this kind of information, or the users demand it, then do all the RESPONSE stuff. Otherwise, avoid it. The email idea is one way of avoiding the whole mess. The redirect to logfile is also interesting; it appears to have many of the same benefits as the email idea. The only detriment is that the client has to keep the browser open to see that the job is completed, and maybe even to see the job results. Jim Penny
participants (9)
-
Ausum Studio -
Casey Duncan -
Charlie Reiman -
Dan Meszaros -
Dieter Maurer -
Eddie Mönch -
Jan Maska -
Jim Penny -
Oliver Bleutgen