[Zope-dev] Zope closes connection if the client closes the write-end of connection

[Izak Burger]

Hi again,

Izak Burger wrote:
> I eventually distilled the check varnish uses into a small C program, 
> and an interesting problem shows up. When you call shutdown(fd, SHUT_WR) 
> on your socket connection, in effect telling zope that you're done 
> talking to it, it looks like zope responds in kind by not talking to you 
> either. I deduce this from the fact that poll() returns POLLHUP in 
> revents and read() returns zero (EOF). POLLHUP, if I understand this 
> correctly, means the other side (zope) has hung up.

It seems this might not be a bug in zope. I'd like to describe what I've 
found this morning and where this is most possibly going wrong. Please 
tell me if I'm mistaken about something.

This goes all the way to what happens when you call poll() on a file 
descriptor. If you ask to be notified for POLLIN events only, it does 
not necessarily guarantee that POLLIN (or a timeout) are the only events 
that can cause poll() to return. The man page does not properly document 
this, but the header file does:

/* Event types always implicitly polled for.  These bits need not be set
    in `events', but they will appear in `revents' to indicate the status
   of the file descriptor.  */
#define POLLERR     0x008       /* Error condition.  */
#define POLLHUP     0x010       /* Hung up.  */
#define POLLNVAL    0x020       /* Invalid polling request.  */

In other words, a POLLHUP event can cause your poll() call to return 
even though the POLLIN event you were looking for didn't occur.

If we inspect Modules/socketmodule.c in the python (2.4.4) source code 
we see that it is assumed that if poll() returns a value greater than 
zero, it means data is available for reading.

If the client calls shutdown(SHUT_WR), this will cause a POLLHUP, which 
fools python into thinking there is data to be read. When it attempts to 
read that data, it ends up with nothing, that is, recv() returns 0.

Moving on to asyncore.py, we see this:

             data = self.socket.recv(buffer_size)
             if not data:
                 # a closed connection is indicated by signaling
                 # a read condition, and having recv() return 0.
                 self.handle_close()
                 return ''

And handle_close eventually ends up telling zope to close the connection.

This has two possible fixes, but both are probably required. Firstly, it 
seems the latest python still has this problem, so I probably need to 
report it as a bug. Secondly, since zope still requires older python 
versions to run, a workaround probably needs to be found. I suppose just 
rebuilding the _socket native module will probably do it and I will test 
this as soon as I have more time.

regards,
Izak