Getting Zope to fetch info via HTTP
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello I have a need to make Zope fetch information from a (slow) external source via HTTP. Would using an ExternalMethod with a call to urllib2 be the right way or would that lock up the thread until the external call returns? Is there in that case any way to allow medusa to do other stuff while the information is not yet available? ...or should I not worry and just increase the number of threads instead? ~ / Johan Lübcke -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6-2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAj3AS/sACgkQ9BeRmxxhT981pgCcCIaJMXlcn30ZxR8ROUFGtqI7 6KMAn3TX84RQI/Nw+vmRhsp7c5HDihox =ObSw -----END PGP SIGNATURE-----
Hi Johan, have you looked at the product KebasData ? This product has an integrated cache function and can parse HTML page for special information. I also heard a product called HTTPMounter or so. Regards, Dirk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dirk Datzert wrote: |have you looked at the product KebasData ? |This product has an integrated cache function |and can parse HTML page for special information. | |I also heard a product called HTTPMounter or so. | Both KebasData and HTTPMounter use httplib to access the external server, but what happens while that request is handled? Will my Zope thread just be waiting idle, or can it handle other requests? ~ / Johan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6-2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAj3AU/oACgkQ9BeRmxxhT99CXgCfRzGSpwHW8RbTrx87qkI/vzJh 1SMAniGqly9yUzPAAyq0BH/TJZSHPuR+ =HIDP -----END PGP SIGNATURE-----
On Thursday 31 October 2002 05:49, Johan Lübcke wrote:
Dirk Datzert wrote: |have you looked at the product KebasData ? |This product has an integrated cache function |and can parse HTML page for special information. | |I also heard a product called HTTPMounter or so.
Both KebasData and HTTPMounter use httplib to access the external server, but what happens while that request is handled? Will my Zope thread just be waiting idle, or can it handle other requests?
~ / Johan
with KebasData, i use timeoutsocket.py to try and limit the time spent on requests. however, this also affects other zope requests. i've looked at TimeoutSocket.py from cvs.zope.org. the operative word here is look :P i havent yet found a good way of tackling this. any info, pointers welcome.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- http://www.kedai.com.my/ http://www.kedai.com.my/eZine http://www.zope.org/Members/kedai http://www.my-zope.org How can I laugh tomorrow, if i can't even smile today.
On Thu, 2002-10-31 at 02:12, Bakhtiar A Hamid wrote:
On Thursday 31 October 2002 05:49, Johan Lübcke wrote:
Dirk Datzert wrote: |have you looked at the product KebasData ? |This product has an integrated cache function |and can parse HTML page for special information. | |I also heard a product called HTTPMounter or so.
Both KebasData and HTTPMounter use httplib to access the external server, but what happens while that request is handled? Will my Zope thread just be waiting idle, or can it handle other requests?
~ / Johan
with KebasData, i use timeoutsocket.py to try and limit the time spent on requests. however, this also affects other zope requests.
i've looked at TimeoutSocket.py from cvs.zope.org. the operative word here is look :P
i havent yet found a good way of tackling this. any info, pointers welcome.
The problem I see is that the timeoutsocket has its own "select" call. So in worst case we wait until the timeout doing nothing. What one would like is to somehow call the poll function in asyncore (The same way the Zope main loop does, so Zope is able to do usefull stuff while we wait). But then, what should the callback be to be able to continue again from the recv statement? (And what happens if we get an exception from something done while we were sleeping...) / Johan
participants (3)
-
Bakhtiar A Hamid -
Dirk Datzert -
Johan Lübcke