Performance requirements and ZEO
I'm working on a project to build a public website using Zope, and am in a position where I need to spec and order the hardware before beginning the work on the site. We're going to plan for 500,000 hits/day as a benchmark. I'm leaning towards a ZEO configuration, with a dedicated ZEO Storage Server and 2 ZEO Client machines, with a load-balancer up front. This way, the worst case is the performance isn't up to par, and I can just toss another machine at it as a ZEO Client. Here's the question: is there any benefit to having dual-processor machines on any of these? I think we can afford 3 of them, if necessary, but is it even needed? I was planning on going with Dual-Xeon 2.8 GHz boxes running Linux. I'd rather not run more than one instance of Zope on the clients, btw. Also, I'm having trouble finding info on getting SSL working in a ZEO setup. Are there any docs out there I should read? much thanks, bart
Depending on the nature of your websites (static content, mostly reads, few reads by content editor) it might be sufficient to have one storage server and two ZEO clients running on a dual CPU maschine plus one maschine as reverse proxy (squid/apache). To be on the safe side add another maschine with some more ZEO clients to your setup. To integrate the load balancer you might choose the follow setup: Internet <---> Proxy <---> Loadbalancer <---> ZEO 1 --> Storageserver <---> ZEO 2 ---> -aj --On Donnerstag, 31. Juli 2003 16:19 Uhr -0400 bart hubbard <bart@barthubbard.com> wrote:
I'm working on a project to build a public website using Zope, and am in a position where I need to spec and order the hardware before beginning the work on the site. We're going to plan for 500,000 hits/day as a benchmark. I'm leaning towards a ZEO configuration, with a dedicated ZEO Storage Server and 2 ZEO Client machines, with a load-balancer up front. This way, the worst case is the performance isn't up to par, and I can just toss another machine at it as a ZEO Client.
Here's the question: is there any benefit to having dual-processor machines on any of these? I think we can afford 3 of them, if necessary, but is it even needed? I was planning on going with Dual-Xeon 2.8 GHz boxes running Linux. I'd rather not run more than one instance of Zope on the clients, btw.
Also, I'm having trouble finding info on getting SSL working in a ZEO setup. Are there any docs out there I should read?
much thanks, bart
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Bart.... I run 2/3 major sites that brings my zope up to about 100,000 hits a day and I can tell you you are going to have a lot of capacity. My set up: - 1.67 AMD - 1.5 GB DDR Ram - 2 x 36 GB SCSI in Raid - Tape backup, etc... I store everything in the ZODB (images, js, css) which totals 59,702 objects. The zope process is fine, it peaks up and down but there are also other sites (minor) running on Apache and mail server running on the machine (also logs, tools, etc etc..). I would say the machine runs at about 3-4% of capacity. I could easily bring this box up to 1,000,000 hits a day without worrying about expanding. Here is some suggestions: - Get lots of RAM. Zope (python) loves RAM (and RAM cache helps) - Up your zope threads (I run at 10) and cache size (10,000) - Get SCSI & RAID (7,200 - 10,000 RPM). Since Zope reads its ZODB from the drive you want this to be fast and reliable. I think you could easily start with one box and migrate up. If you are planning on a dual, look into Python and SMP. I remember reading it was not great, so what you could do is run two instances (your ZEOs) one on each processor. Cheers, BZ
I'm working on a project to build a public website using Zope, and am in a position where I need to spec and order the hardware before beginning the work on the site. We're going to plan for 500,000 hits/day as a benchmark. I'm leaning towards a ZEO configuration, with a dedicated ZEO Storage Server and 2 ZEO Client machines, with a load-balancer up front. This way, the worst case is the performance isn't up to par, and I can just toss another machine at it as a ZEO Client.
Here's the question: is there any benefit to having dual-processor machines on any of these? I think we can afford 3 of them, if necessary, but is it even needed? I was planning on going with Dual-Xeon 2.8 GHz boxes running Linux. I'd rather not run more than one instance of Zope on the clients, btw.
Also, I'm having trouble finding info on getting SSL working in a ZEO setup. Are there any docs out there I should read?
much thanks, bart
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On Thu, Jul 31, 2003 at 04:23:39PM -0500, BZ wrote:
I think you could easily start with one box and migrate up. If you are planning on a dual, look into Python and SMP. I remember reading it was not great, so what you could do is run two instances (your ZEOs) one on each processor.
That is correct. Read this: http://www.zope.org/Members/glpb/solaris/report_ps -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's THE TOFU RAT! (random hero from isometric.spaceninja.com)
one more thing... On Thu, Jul 31, 2003 at 04:23:39PM -0500, BZ wrote:
- Up your zope threads (I run at 10) and cache size (10,000)
If you boost the zope threads you probably also want to boost the number of zodb connections, per the Zope Book: http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/MaintainingZope.... "...the number of database connections should always be higher than the number of ZServer threads by a few..." Also, here's another doc about optimization; it's written for a Plone audience but many/most of the lessons apply to any zope app. http://plone.org/documentation/book/9 and another... http://www.zettai.net/Howto/zopecaching/view -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's PROCTO-GOD! (random hero from isometric.spaceninja.com)
I wrote:
If you boost the zope threads you probably also want to boost the number of zodb connections, per the Zope Book: http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/MaintainingZope....
"...the number of database connections should always be higher than the number of ZServer threads by a few..."
... hmm, this seems to be particular to running without ZEO. If running with ZEO, there's a parameter to ClientStorage that may be of interest: cache_size -- The disk cache size, defaulting to 20 megabytes. This is passed to the ClientCache constructor. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's PEDOPHOBIAC FROM THARG! (random hero from isometric.spaceninja.com)
Paul Winkler wrote:
one more thing...
On Thu, Jul 31, 2003 at 04:23:39PM -0500, BZ wrote:
- Up your zope threads (I run at 10) and cache size (10,000)
If you boost the zope threads you probably also want to boost the number of zodb connections, per the Zope Book: http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/MaintainingZope....
"...the number of database connections should always be higher than the number of ZServer threads by a few..."
[snip] I'd like to add that this only applies if you are mainly serving from the ZODB; if your data comes from an external source, for instance an RDBMS, it might make sense to leave the ZODB connections at the default and only increase the number of ZServer threads since they aren't going to hit the ZODB much anyway. - peter.
Here's the question: is there any benefit to having dual-processor machines on any of these? I think we can afford 3 of them, if necessary, but is it even needed? I was planning on going with Dual-Xeon 2.8 GHz boxes running Linux. I'd rather not run more than one instance of Zope on the clients, btw.
Even if Python threads *loved* multiple processors, I usually find SMP to be less than optimal. You can do the math for yourself when you get your quotes, remembering that two processors will compute significantly below 200% of a single processor no matter what you do. My strategy: Get a good solid server for your ZEO server. Run the ZEO client on this one too, and see if you can get away with it. If not, add cheapo web servers with ZEO clients until you're comfortable with the load. If you're clever, these might not even need disks. (See Knoppix.) Sure, they aren't serious machines, and they might fail. But who cares? You have bunches of them! If you run Zope on a mulitple processor system, run one ZEO client per processor, and set processor affinity if you can. (Solaris can, and so can newer Linux kernels .) You may not like this much, but it's not so bad if you use zopectl or something like it. And remember, Zope will benefit a LOT more from extra memory than from extra processors. --jcc -- "My point and period will be throughly wrought, Or well or ill, as this day's battle's fought."
participants (6)
-
Andreas Jung -
bart hubbard -
BZ -
J Cameron Cooper -
Paul Winkler -
Peter Sabaini