[ZODB-Dev] performance of chunked data over ZEO

[Jeremy Hylton]

On Mon, 29 Nov 2004 22:06:41 -0500, Tim Peters <tim at zope.com> wrote:
> > Increasing the chunk size to 64K caused a performance loss to about
> > 100-150 KB/seconds. Any ideas why increasing the chunk size causes such a
> > dramatic performance loss?
> 
> Not really; best guesses are that one case is exploiting a cache the other
> case isn't, or that it will resolve to some inscrutable detail of how socket
> TCP is implemented on your (unidentified) platform.  For example, whenever
> TCP figures vary wildly for no apparent reason, speculation about Nagle's
> algorithm is traditional.  Jeremy tried to cut that off by having ZEO do its
> own layer of packet chunking, but who knows.

The specific interaction between Nagle's algorithm and delayed
acknowledgements is easy to observe using a tool like tcpdump.  You
see small packets going over the wire, a long pause, then another
small packet.  In general, if you have a specific benchmark, observing
the packets on the wire goes a long way.  There's a ton of inscrutable
software between the wire and the app.  If you can work backwards from
the actual packets on the wire, it helps to explain what the layers
above are doing.

Jeremy