[Zope] Optimisation fun and games

[Richard Jones]

We (http://www.ekit.com/) have a popular website, and it's hit fairly hard 
(around 550k hits since the start of May, and with the summer holidays just 
starting, we're expecting it to skyrocket). I've therefore been working on 
ways to get Zope to go as fast as possible. Here's some hints for others who 
are running (or are expecting to run) high-volume Zope sites:

1) ZEO clients - we have about 9 of them, spread over a number of CPUs. We'll
   be putting in some more real soon.
2) Two apache frontends using FastCGI to the ZEO clients. They have an on-disk
   cache which serves up images (using rewrite rules to trap image URLs before
   they are sent to Zope) meaning Zope can do more important stuff than serve
   up static binary files. This is currently in testing and halves the average
   page load time. We don't have a real loadbalancer set up for the apaches,
   we just serve up different domains (we have many) from each. Watch the max
   number of connections per apache though - we've had to up it several times.
3) Several uses of RAM caches in Zope - the stylesheet, the sidebar HTML chunk
   and the front page content HTML chunk. Use the CallProfiler to isolate 
   calls that change very little between users. RAM cache those.
4) CallProfiler'ed up the wazoo. Lots of CallProfiler analysis and fixing of
   hotspots. We managed halve our average page load time just doing this.

Just thought this might be interesting :)

One very interesting thing to note though is that the webmail part of our 
service is done in PHP. It's running on the same machines as the apache 
frontends. It doesn't seem to struggle at all. Not that we'd ever dream of 
writing something as complex as ekit in PHP :)


   Richard