Re: [Zope] Zope URL/Page Retrieval with Python
Samir Mishra <SamirMishra@cbuae.gov.ae> wrote ..
The Zope site I'm trying to access is on another server. I'm accessing it remotely, so I can't use external methods. I wanted to access the web page and parse the HTML for relevant info.
And I found the answer to my question on another mailing list. If anyone's interested --
import urllib
URLsite = "http://username:password@zserver:8080/everything/else/on/path"
usock = urllib.urlopen(URLsite) data = usock.read() usock.close()
Someone else please correct me if I am wrong but we decided not to use something similar because if for some reason the site you are trying to urlopen is down, etc. then it never really times out...just sits and spins and locks that zope thread. Do that a few times and your whole site is down. Not sure if a try: would work here but that might be worth a shot. But I guess if the urlopen never comes back then the try: would just try forever. Am I right here or is there a better way to handle this? Allen
aschmidt@fredericksburg.com wrote:
Samir Mishra <SamirMishra@cbuae.gov.ae> wrote ..
import urllib
URLsite = "http://username:password@zserver:8080/everything/else/on/path"
usock = urllib.urlopen(URLsite) data = usock.read() usock.close()
Someone else please correct me if I am wrong but we decided not to use something similar because if for some reason the site you are trying to urlopen is down, etc. then it never really times out...just sits and spins and locks that zope thread. Do that a few times and your whole site is down.
Not sure if a try: would work here but that might be worth a shot. But I guess if the urlopen never comes back then the try: would just try forever.
Am I right here or is there a better way to handle this?
Allen
There is a module available called timeoutsocket.py which enables timeouts for all TCP connections. Last time I checked, the site was down: http://www.timo-tasi.org/python/timeoutsocket.py, but the script was available by going through archive.org. I believe the module has been included in python 2.3 as part of the standard distribution.
Samir Mishra <SamirMishra@cbuae.gov.ae> wrote ..
The Zope site I'm trying to access is on another server. I'm accessing it remotely, so I can't use external methods. I wanted to access the web page and parse the HTML for relevant info.
And I found the answer to my question on another mailing list. If anyone's interested --
import urllib
URLsite = "http://username:password@zserver:8080/everything/else/on/path"
usock = urllib.urlopen(URLsite) data = usock.read() usock.close()
<shameless plug> you could also try a zope product that does all that. get KebasData at http://www.zope.org/Members/kedai/KebasData -it has a timeout (default 15s) using timeoutsocket.py (will probably update that once zope runs pythn2.3) -it can parse anything as you want </shamless plug> feedback welcome.
On Sat, Nov 15, 2003 at 07:42:51AM -0500, aschmidt@fredericksburg.com wrote:
import urllib
URLsite = "http://username:password@zserver:8080/everything/else/on/path"
usock = urllib.urlopen(URLsite) data = usock.read() usock.close()
Someone else please correct me if I am wrong but we decided not to use something similar because if for some reason the site you are trying to urlopen is down, etc. then it never really times out...just sits and spins and locks that zope thread. Do that a few times and your whole site is down.
Right, if you are connecting from zope to another server. However, I think the original poster was connecting to zope from a plain old python program on a non-zope system.
Not sure if a try: would work here but that might be worth a shot. But I guess if the urlopen never comes back then the try: would just try forever.
yes it would. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's PSEUDO BUSHY CHAIR KING LEAR ! (random hero from isometric.spaceninja.com)
participants (4)
-
aschmidt＠fredericksburg.com -
Bakhtiar A Hamid -
Jay Dorsey -
Paul Winkler