[Zope] Integrating Zope with Excel.

Jean-Francois.Doyon at CCRS.NRCan.gc.ca Jean-Francois.Doyon at CCRS.NRCan.gc.ca
Thu Sep 2 11:55:00 EDT 2004


I've had success with setting the timeout.  The only thing that bugs me
right now is that I can only set it globally.

Right now I have a monkey patch that fixes up httplib with a default
timeout, which makes it global to any httplib
based request (such as urllib and urllib2 functionality).  I also did the
same with FTP.

Now at least I don't worry that some bad request will lock up a thread for
an inordinate amount of time, potentially forcing
zope to eventually run out of threads.

I really whish the Python API for socket based functionality would've
followed to allow specifying the timeout for each
request though, in fact I find it outright bizarre it didn't!! Foe exmaple
you can't specify a timeout for a request done
through urllib directly, since you can't have access to the socket before
the request goes out.

Anyways, all this to say that the problem of having Zope generating network
connections and locking up threads is evry real,
and given the current design, makes sense.

I've been toying with the idea of implementing a framework that would fork
threads or start a seperate process for executing
these types of requests, and providing a consistent API for Zope, allowing
things to be asynchronous, and protecting the core thread pool.

In the world of web services, this would be most handy.  Of course the
performance impact vs. benefits would have to be studied.

No matter, it's more work than I'm willing to take on though .. Maybe in
Zope 3?

J.F.

-----Original Message-----
From: zope-bounces at zope.org [mailto:zope-bounces at zope.org]On Behalf Of
Paul Winkler
Sent: September 2, 2004 11:38 AM
To: zope at zope.org
Subject: Re: [Zope] Integrating Zope with Excel.


On Thu, Sep 02, 2004 at 09:32:54AM +0100, Chris Withers wrote:
> Paul Winkler wrote:
> >I was using either urllib or urllib2 (don't remember).
> >At that time (python 2.1), socket.setdefaulttimeout() did not
> >exist so any request to a socket could potentially spin forever.
> 
> Cool, I didn't know python had finally grown one of those. Yay!
> 
> >My theory is that, each time a request came for my 
> >external method, the current worker thread was then
> >waiting forever for a reply from the firewalled socket.
> >So, inevitably, sooner or later all threads were doing this
> >and the server was effectively hung.
> 
> That makes sense...
> 
> >But, I haven't been able to reproduce this behavior today with Zope
2.7.2,
> >even with socket.setdefaulttimeout(None).
> >I've currently got 16 pending requests for the never-finishing
> >external method, and I can still browse zope just fine.
> 
> ...but this doesn't. Have you upped the number to work threads to > 16?

Nope. This was a 2.7.2 server with the default 4 threads. 

Btw, I notice that even with socket.setdefaulttimeout(None),
the requests eventually timed out after some minutes :-)
Don't know how many, I didn't keep track.
 
-- 

Paul Winkler
http://www.slinkp.com
_______________________________________________
Zope maillist  -  Zope at zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


More information about the Zope mailing list