[Zope-dev] Re: [Zope] [BUG] Background processes interfere with Zope's HTTP responses

[Steve Spicklemire]

Hi Deiter,

   Hmm.. I just tried this with my highly experimental product
(LocalProc 0.0.1) and I didn't observe the same effect that you did
behind my Proxy Server. (I normally run with Rewrite... but when I got
your note I tried ProxyPass and it worked on the first try.)

(Good howto Anser! http://www.zope.org/Members/anser/apache_zserver/)

Anyway... I ran 'sleep 20' as my local process (I checked the
'background' button on my product) and the web page returned
immediately, but the process ran on in the background as I wished. The
main difference I see between what you are doing and what I do.. is
that I redirect stdout and stderr so that my subprocess has no open
files shared with the parent process.. anyway.. you might try this to
see if it suits your needs. Lightly tested on FreeBSD only.. but any
unix should work the same...

http://www.zope.org/Members/sspickle/LocalProc

-steve

>>>>> "Dieter" == Dieter Maurer <dieter@handshake.de> writes:

    Dieter> Today was again a hard Zope day.

    Dieter> Did you ever hear, that usually (i.e. if you do not do
    Dieter> special things) Zope renders a complete page before it
    Dieter> starts sending the result back to the client.  This
    Dieter> implies that you should never see only half of a page.

    Dieter> I heard it and I saw the code that does it.  Therefore, I
    Dieter> was convinced that this were really the case.

    Dieter> Then our client phoned:

    Dieter>       He visits a page that allows him to generate a
    Dieter> newsletter. When he presses the "generate newsletter
    Dieter> button", a result page is build that tells him that the
    Dieter> newsletter is being generated and send is a few minutes.
    
    Dieter>       He reported, that this response page is build only
    Dieter> half, then stops, to be completed only half a minute
    Dieter> later, at the same time when the newsletter arrives.  The
    Dieter> effect is reproducible. If newsletter generation takes
    Dieter> longer, he must wait longer for the page to complete.
    
    Dieter>       He is convinced that we generate the newsletter
    Dieter> synchronously and let him wait until the generation is
    Dieter> complete.  And he is angry.
    
    Dieter>       I know, that the newsletter is generated in a
    Dieter> background process, started in an external method with:
    
    Dieter>     	   os.system("gen_newsletter &")
    
    Dieter>       I try to reproduce the behaviour in our test
    Dieter> environment and fail. I do not have to wait, until the
    Dieter> newsletter generation finished. Then, I use the official
    Dieter> service URL and see, I observe the same behaviour.  The
    Dieter> difference: in the test environment, the browser connects
    Dieter> directly to ZServer; with the official URL, it connects
    Dieter> via a proxy.
    
    Dieter>       Puzzling!


    Dieter> What happened?

    Dieter>      The background process inherits Zope's open file
    Dieter> descriptors.  Among them are all currently open HTTP
    Dieter> request sockets.  The process keeps these sockets open
    Dieter> until it finishes.

    Dieter>      There are two HTTP modes:

    Dieter>        1. single request mode a new TCP connection is
    Dieter> created for each HTTP request, the request is completed,
    Dieter> when the TCP connection is closed.

    Dieter>        2. multi request mode several requests share a
    Dieter> single TCP connection.  The "Content-Length" HTTP header
    Dieter> allows client and server to determine the request
    Dieter> boundaries.

    Dieter>      That means: when a client uses the multi request
    Dieter> mode, everything is fine. Clients, however, that use the
    Dieter> single request mode wait until their connection is closed
    Dieter> and can observe serious delays.

    Dieter>      These delays may be very difficult to explain, as the
    Dieter> background process not only delays its own request but may
    Dieter> delay arbitrary other requests that happen to be served at
    Dieter> the same time.



    Dieter> How to fix the problem:

    Dieter>       There should be a (file) control "CloseOnExec", that
    Dieter> tells Unix to automatically close the file like object
    Dieter> upon an exec.  ZServer should probably use it on each of
    Dieter> its sockets.  I can not see a serious application that
    Dieter> should have direct access to ZServer's HTTP request
    Dieter> socket.


    Dieter> I will soon file a bug report into the Collector.


    Dieter> Dieter

    Dieter> _______________________________________________ Zope
    Dieter> maillist - Zope@zope.org
    Dieter> http://lists.zope.org/mailman/listinfo/zope ** No cross
    Dieter> posts or HTML encoding!  ** (Related lists -
    Dieter> http://lists.zope.org/mailman/listinfo/zope-announce
    Dieter> http://lists.zope.org/mailman/listinfo/zope-dev )