Hi, can one interrupt the execution of a script when a certain amount of time has passed? What I want to do: - I start with a given URL. - Then I read the webpage denoted by it into a string like page=urllib.urlopen(URL).read() and if it satisfies a given condition (e.g., it contains a certain word), parse it to find out all the urls it contains. - With these new urls I continue - I take care not to read the same URL twice This works fine in general, but there are certain web pages that never respond. If you try view them with a browser you get a timeout after a few mintues. But the urllib.urlopen() script keeps waiting until it gets a response and if never gets one, the whole program is stopped. What I would like now is to interrupt the script an return '' if it gets no response after 10secs. Any idea??? Horst _________________________________________________________________ Downloaden Sie MSN Explorer kostenlos unter http://explorer.msn.de/intl.asp
Horst Wald writes:
can one interrupt the execution of a script when a certain amount of time has passed? Not easily...
Signals would be used for such asynchronous events. They have lots of restrictions in a multi-threaded environment such as Zope (they do not work under most Unix implementations in this context).
... timing out "urllib.urlopen" ... It may depend on your operating system, whether this is possible or not. However, you will need to work a bit to get it:
It is probably best to use "urllib2" and provide your own HTTP handler. There is a timeout enhanced "socket" module for Python, available for some platforms. Search the "comp.lang.python" archives for it. If you can use it for your platform, then it is quite easy. Enhance "httplib" by using the timeout enabled "socket" module. Make your own "urllib2.HTTPHandler" to use the enhanced version of "httplib" and pass your timeout value. Otherwise, you may try to use non-blocking socket operations and control the timeouts via "select". In this case, you will probably need to implement "urlopen" yourself as the used "socket.makefile" will not use a timeout controlled "select". Dieter
participants (2)
-
Dieter Maurer -
Horst Wald