[ZODB-Dev] ZEO overhead?

[Shane Hathaway]

On 08/01/2003 01:04 PM, Neil Schemenauer wrote:
> I'm considering ripping the asyncore parts out of the ZEO client and
> server, making them both synchronous, and seeing how much difference
> that makes.  Does anyone have a better idea of how to track down this
> problem (if it is indeed a problem at all)?

I'd be quite happy to see someone try this experiment.  The ZEO client 
is fundamentally synchronous, yet it goes to the trouble of switching 
threads every time it talks to the storage server, just so it can 
support asynchronous communication.  Python's GIL makes us pay a high 
penalty for switching threads.  To make things worse, the asyncore map 
contains both the ZEO connections and the connections with the world. 
So asyncore has to wade through all of the incoming connections just to 
talk to the ZEO server.

In reality, I think there's little reason why ZEO over a local 
connection should be noticeably slower than using FileStorage directly. 
  Network communication time should be dwarfed by hard drive seek time.

It shouldn't be especially difficult to switch to synchronous 
communication.  The one thing you'll have to be aware of is that ZEO 
does make use of asynchronous communication to receive invalidation 
messages.  You'll need to get the client to poll frequently for incoming 
invalidations.

Shane