Optimisation fun and games
We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around 550k hits since the start of May, and with the summer holidays just starting, we're expecting it to skyrocket). I've therefore been working on ways to get Zope to go as fast as possible. Here's some hints for others who are running (or are expecting to run) high-volume Zope sites: 1) ZEO clients - we have about 9 of them, spread over a number of CPUs. We'll be putting in some more real soon. 2) Two apache frontends using FastCGI to the ZEO clients. They have an on-disk cache which serves up images (using rewrite rules to trap image URLs before they are sent to Zope) meaning Zope can do more important stuff than serve up static binary files. This is currently in testing and halves the average page load time. We don't have a real loadbalancer set up for the apaches, we just serve up different domains (we have many) from each. Watch the max number of connections per apache though - we've had to up it several times. 3) Several uses of RAM caches in Zope - the stylesheet, the sidebar HTML chunk and the front page content HTML chunk. Use the CallProfiler to isolate calls that change very little between users. RAM cache those. 4) CallProfiler'ed up the wazoo. Lots of CallProfiler analysis and fixing of hotspots. We managed halve our average page load time just doing this. Just thought this might be interesting :) One very interesting thing to note though is that the webmail part of our service is done in PHP. It's running on the same machines as the apache frontends. It doesn't seem to struggle at all. Not that we'd ever dream of writing something as complex as ekit in PHP :) Richard
Hello Richard!
We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around 550k hits since the start of May, Do you mean hits (images, pages, everything) or page impressions?
Greetings Sven
On Mon, 20 May 2002 12:14, Sven Rudolph wrote:
Hello Richard!
We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around 550k hits since the start of May,
Do you mean hits (images, pages, everything) or page impressions?
It's an approximation of number of visits - ie. the number of hits on our front page HTML. Richard
Do you mean hits (images, pages, everything) or page impressions?
It's an approximation of number of visits - ie. the number of hits on our front page HTML. 550,000 visits? You mean visits? Well that's plenty. If you assume 3-4 page impressions per visit, you get 2 million page impressions. I found 128 images on your frontpage. Including the page itself, this is 129 hits per page impression. This results in 258,000,000 hits in 15 days. I think 9 ZEO clients are really needed in this case ;-)
I wish we had as much visits as you have... Greetings Sven
On Mon, 20 May 2002 12:32, Sven Rudolph wrote:
Do you mean hits (images, pages, everything) or page impressions?
It's an approximation of number of visits - ie. the number of hits on our front page HTML.
550,000 visits? You mean visits? Well that's plenty. If you assume 3-4 page impressions per visit, you get 2 million page impressions. I found 128 images on your frontpage. Including the page itself, this is 129 hits per page impression. This results in 258,000,000 hits in 15 days. I think 9 ZEO clients are really needed in this case ;-)
That estimate is actually a bit high due to the client-side caching of images. We have a number (just under 100) of domains with their own access logs. Taking the most popular domain in May, www.ekno.lonelyplanet.com, we have around 200k visits resulting in around 12 million hits. That accounts for about half the traffic we see (and one of the apaches I mentioned - all other domains go through the second apache). Richard
Richard Jones wrote:
2) Two apache frontends using FastCGI to the ZEO clients.
Is FastCGI doing you any favours here? Have you tried ProxyPassing?
we just serve up different domains (we have many) from each. Watch the max number of connections per apache though - we've had to up it several times.
Have a look at using Squid instead of Apache, it's what ZC have done in siutations like this...
One very interesting thing to note though is that the webmail part of our service is done in PHP. It's running on the same machines as the apache frontends. It doesn't seem to struggle at all. Not that we'd ever dream of writing something as complex as ekit in PHP :)
Indeed. Do you use ZPT? If so, join the lets-optimise-ZPT campaign ;-) cheers, Chris
On Mon, 20 May 2002 18:17, Chris Withers wrote:
Richard Jones wrote:
2) Two apache frontends using FastCGI to the ZEO clients.
Is FastCGI doing you any favours here? Have you tried ProxyPassing?
*shrug* - I inherited it.. Anthony? :)
we just serve up different domains (we have many) from each. Watch the max number of connections per apache though - we've had to up it several times.
Have a look at using Squid instead of Apache, it's what ZC have done in siutations like this...
The apache serves up a bunch of other stuff - and as Anthony's mentioned, almost none of our pages are static.
One very interesting thing to note though is that the webmail part of our service is done in PHP. It's running on the same machines as the apache frontends. It doesn't seem to struggle at all. Not that we'd ever dream of writing something as complex as ekit in PHP :)
Indeed. Do you use ZPT? If so, join the lets-optimise-ZPT campaign ;-)
Nope, nothing quite so new-fangled as that... large amounts of the dtml have the old dtml syntax too :) Richard
On Mon, 20 May 2002 18:17, Chris Withers wrote:
Richard Jones wrote:
2) Two apache frontends using FastCGI to the ZEO clients.
Is FastCGI doing you any favours here? Have you tried ProxyPassing?
Oh, quickie question - what sort of performance difference is there? Richard
Richard Jones wrote:
On Mon, 20 May 2002 18:17, Chris Withers wrote:
Richard Jones wrote:
2) Two apache frontends using FastCGI to the ZEO clients.
Is FastCGI doing you any favours here? Have you tried ProxyPassing?
Oh, quickie question - what sort of performance difference is there?
Dunno, I've just never conceptually got on with FastCGI, how ever small the CGI stub is, you're still reading a file from disk and executing it for every single request... cheers, Chris
Chris Withers wrote:
Richard Jones wrote:
2) Two apache frontends using FastCGI to the ZEO clients.
Is FastCGI doing you any favours here? Have you tried ProxyPassing?
Not related to performance, but here using fastcgi I don't seem to have the problems with cookie authentication that has been reported with apache 1.3.23 (unless it has been fixed in the mandrake's rpm, I didn't test yet with proxypass) Bye -- Luca Olivetti Wetron Automatización S.A. http://www.wetron.es/ Tel. +34 93 5883004 Fax +34 93 5883007
On Monday 20 May 2002 09:29 am, Richard Jones wrote:
We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around 550k hits since the start of May, and with the summer holidays just starting, we're expecting it to skyrocket). I've therefore been working on ways to get Zope to go as fast as possible. Here's some hints for others who are running (or are expecting to run) high-volume Zope sites:
1) ZEO clients - we have about 9 of them, spread over a number of CPUs. We'll be putting in some more real soon. 2) Two apache frontends using FastCGI to the ZEO clients. They have an on-disk cache which serves up images (using rewrite rules to trap image URLs before they are sent to Zope) meaning Zope can do more important stuff than serve up static binary files. This is currently in testing and halves the average page load time. We don't have a real loadbalancer set up for the apaches, we just serve up different domains (we have many) from each. Watch the max number of connections per apache though - we've had to up it several times. 3) Several uses of RAM caches in Zope - the stylesheet, the sidebar HTML chunk and the front page content HTML chunk. Use the CallProfiler to isolate calls that change very little between users. RAM cache those. 4) CallProfiler'ed up the wazoo. Lots of CallProfiler analysis and fixing of hotspots. We managed halve our average page load time just doing this.
Just thought this might be interesting :)
i'd suggest 5 - squid. throw in squid and stuff will be served much much faster. off course, the pages need to be cacheable i.e with at least Expires field, or better yet put out the necessary cache headers. -- http://www.kedai.com.my/ http://www.kedai.com.my/eZine http://www.zope.org/Members/kedai http://www.my-zope.org We don't need no, no no no, no parental guidance here
On Mon, 20 May 2002 19:39, Bakhtiar A Hamid wrote:
On Monday 20 May 2002 09:29 am, Richard Jones wrote:
We (http://www.ekit.com/) have a popular website, and it's hit fairly hard (around 550k hits since the start of May, and with the summer holidays just starting, we're expecting it to skyrocket). I've therefore been working on ways to get Zope to go as fast as possible. Here's some hints for others who are running (or are expecting to run) high-volume Zope sites:
1) ZEO clients - we have about 9 of them, spread over a number of CPUs. We'll be putting in some more real soon. 2) Two apache frontends using FastCGI to the ZEO clients. They have an on-disk cache which serves up images (using rewrite rules to trap image URLs before they are sent to Zope) meaning Zope can do more important stuff than serve up static binary files. This is currently in testing and halves the average page load time. We don't have a real loadbalancer set up for the apaches, we just serve up different domains (we have many) from each. Watch the max number of connections per apache though - we've had to up it several times. 3) Several uses of RAM caches in Zope - the stylesheet, the sidebar HTML chunk and the front page content HTML chunk. Use the CallProfiler to isolate calls that change very little between users. RAM cache those. 4) CallProfiler'ed up the wazoo. Lots of CallProfiler analysis and fixing of hotspots. We managed halve our average page load time just doing this.
Just thought this might be interesting :)
i'd suggest
5 - squid. throw in squid and stuff will be served much much faster. off course, the pages need to be cacheable i.e with at least Expires field, or better yet put out the necessary cache headers.
Yes - we've had a lot of fun coercing stuff so it's cacheable. So on to 6: 6. make stuff cacheable. Put stylesheets and images in an area of your Zope that isn't authenticated by cookies. Most browsers won't cache a stylesheet if a cookie was sent along with the request. Similarly, don't request stuff by HTTPS when you don't need to - not only does the server have to do more work, but the client won't cache anything served by HTTPS. There's a neat tool out there that does cacheability testing: http://www.mnot.net/cacheability/ Richard
participants (5)
-
Bakhtiar A Hamid -
Chris Withers -
Luca Olivetti -
Richard Jones -
Sven Rudolph