"Michael A. Alderete" wrote:
I have scanned through the ZAG, the various how-tos and tips on Zope.org, and searched the archives of this mailing list, but I haven't found anything which puts into writing a concrete methodology for optimizing performance of Zope.
If there is one and I'm just too stupid to find it, I'd love to get a pointer. If there isn't one I'll write it, if people will send me information and teach me the zen of optimal Zope performance.
Here are the kinds of things I would expect to see in this kind of document:
1. Performance-oriented Platforms
* Which platform (hardware / OS combination) is the best for Zope?: - if money is no object
Alpha 21264, high MHz, lots of ram and CPUs, running on Fiberchannel RAID with a quad port gigabit ethernet card, running Tru64 Unix (pardon me whilst I wipe the drool from my keyboard ...).
- if money is the main object See below ;-)
- if it has to run on a cheap PC, 'cause that's what people have - etc.
Linux on a celeron, or dual celeron.
Questions this needs to answer:
- Do multiple processors help?
Yes, if the OS can scale well with them. IE, CPU >=2 is a waste with Windows9X.
- How much RAM?
As much as you can. Seriously. If you intend to run a really high-traffic site, and or a big ZODB, you can never (again, with the exception of win9x) have too much RAM.
- Disk subsystem recommendations / tradeoffs
RAID where possible. Personally, I favor RAID 0+1 (aka RAID 10, iirc) or RAID 5. RAID 1 is mirroring, providing redundancy, tradeoff: write speed is slower (writing to two disks, not one), but under some conditions can be nealy 2x as fast for reads (interleving of the reading) RAID 0 is basically like srtiping, you get N drives, and they become one big drive, with roughly N(Drive Speed) performance. Tradeoff: one drive fails and you better have a backup. IDE is cheap, but IMO, is a bad choice for a production server.
- Solaris vs. Linux vs. FreeBSD [vs. Windows NT?] ...
(I now have a 10 month old daughter sleeping on my shoulder, so please forgive any typos thatt slip through :-) Here, money can be an issue. Obviously, your chepest bet would be a PC ruuning one of the free unix-likes (personally, I'd avoid solaris whenever possibly, YMMV). From my experience, Linux (and likely *BSD) will run circles around an NT machine of the same configuration for servers.
- etc...
2. Performance-oriented Configurations
* What is the right way to set up Zope?:
Will vary by personal tastes and needs.
- which web server to use
From what I have heard, run at great spread from NES.
I don't -yet- have Zope+Zeus experience (nor have I heard any), so I can't comment on it. Straight ZServer, if all you need is Zope, go for it. I did this (and for some still do) for my sites. Apache: Well, the thing to remember about Apache was it was designed for extensibility, not speed. Not to imply it is a slouch by any means. With Apache, there are many things you can do outside of Zope. Roxen: my current favourite. Roxen is very, very cool. Another option I don't know whether or not has been tried, would be to run a squid/apache proxy as a front-machine as an accellerator. Anyone done this yet? I am considering trying a similiar setup with mirroring Roxen servers.
- which CGI configuration to use
Depends on needs. Assuming here you are talking about using Zope w/another server. I understand the FastCGI support is maturing nicely, but have not yet tried it out (anyone done it with Roxen?). Standard PCGI works well enough for me so far.
- which ZODB storage system to use
Depends on how deep you want to get into code :-)
- etc.
Questions this needs to answer:
- ZServer vs. Apache [vs. IIS?] ...
Needs dependent. IMO, for a site that will be pure Zope, any other server (outside the accellerated option) is a waste.
- PCGI vs. FastCGI [vs. ??]
IMO, this boils down to stability, and traffic.
- ZODB in file system vs. database
Part of this is dependant on the filesystem. yes, I refer to the x86 2Gb file limit in the standard Linux install. There are patches to dissolve the issue, an done can use an Alpha machine, where the issue is non-extant.
- which database?
Another personal and needs based choice. Many have (IMO, correctly) indicated that MySQl would be inadequate for using as a ZODB replacement, due to transactions being non-extant. As far as otherwise, an RDMS really needs to be a choice brought about by a proper needs analysis.
- etc.?
3. Performance Tuning
I don't know where to even start asking questions here, but I read a cursory mention that zope.org was recently sped up considerably by playing with some caching parameters, or something like that. So detailed tuning suggestions are needed for:
- operating system - web server - Zope - ??
I haven't felt the need yet to do much tuning here, but I do have some general suggestions. IIRC, the caching tuning you mentioned on zope.org was an increase in the amount of objects kept in cache. IMO, You need to correlate how much RAM you can spare, and how much you want kept in Zope's cache. if you have limited RAM, you probably don't want to have a high number. A machine in swap is not a snappy machine. IMo, most of the truly useful gains in a clean slate would be the hardware and OS setup in general. More RAM, more MHz, and fast disks will go a long way to making your serve rmore responsive. On that note, there is a little tip for Linux users I ran into somewhere (where was it?) about doing a chmod +t on an executable file would keep it in memory. I have not tried this with Zope yet, but it does work nicely on other executables (learned this from a Sybase HPUX tip IIRC) Can't really say much about the rest, yet. I think it is to general to really do justice with (an d this is getting long :-). I will mention that I have or am running zope on the following platforms: o Alpha 4/233 48M Ram (RAM was a problem here) o Intel p133, 128M RAM (multiple Zope instances ~4, no complaints) o Dual Celeron 450s, 256MB RAM, RAID0 (~50MB/s) (FAST!) o Dual PII 300s, 256MB RAM, HP AutoRAID cabinet (~80MB/s) (FAST!) o HP C3000 Visualize, 1.5GB RAM (FAST!) o HP N-Class 4+ processors, several Gig of RAM (Sit down, strap in, hold on and shut up!) I haven't run any standardized benchmarks against them yet, but I plan on doing so for the Dual CPU machines, and the two HP babies. On the latter, I am checking to see if my boss will let me publish my results on them externally (wouldn't know why they wouldn't ... :). Anybody have a preferred webserver benchmark that is appropriate for Zope? If desired, I could turn this into a howto ... Bill -- In flying I have learned that carelessness and overconfidence are usually far more dangerous than deliberately accepted risks. -- Wilbur Wright in a letter to his father, September 1900