[Zope] How many hits can zope handle?

Chris McDonough chrism@zope.com
07 Jul 2003 23:39:56 -0400


Maybe. 

That's a lot of traffic.  Ten billion hits per day equates to 115,740
hits per second of continuous steady-state traffic (10,000,000,000 /
86400).  A typical rule of thumb for system scaling is that 5 * nonpeak
hits/sec needs to be accounted for in the design, so you'd probably need
to design for about 580,000 hits/sec to meet peak load. 

If you can offload 90% of the traffic to a caching tier (very possible
if you don't need to do many writes), that means every 1 out of 10
requests or so will make it to a Zope appserver.  You can also assume
that a single cache server on Intel hardware (you'll need to convert
this to mainframe terms) can handle about 600 requests per second. 
Thus, your application will require about 966 cache server instances
(580,000 / 600). 

If 10% of the traffic makes it to Zope, this means that the Zope tier
will need to service about 58,000 requests per second.  If you design
your application properly, you can expect a hit rate from a single Zope
instance on Intel hardware of about 50 - 90 requests/sec.  This means
you're going to need at least 644 appservers (58,000 *.1 / 90).  I don't
know how many ZEO servers this setup would need, we've never needed to
project ZEO server scale like that.  Probably about 50. 

FWIW, just to get an idea of the scale of something to handle this:  

- Your estimated bandwidth requirements for such a setup will be 
  about 1160 Mbits/s during peak load (if you assume each hit 
  consumes about 250 bytes to account for overhead, which is a very 
  low estimate). 

- If a T1 costs US $700 a month, you'll be spending ~$525,000.00 a 
  month (1160 / 1.544 * 700) retail just for the connectivity charges 
  to meet your peak demand. 

- The initial hardware investment (if you need it) will cost about 
   US$3.5 million+ (1660 servers * 2100/apiece). 

- If you figure that one competent sysadmin can administer about 60 
  machines, you'll likely need to hire around 26 full time admins to 
  keep the system going. 

Given all that, are you sure about your hit projection? 

- C 


On Mon, 2003-07-07 at 21:40, Wayne Connolly wrote: 
> Can Zope handle 10 billion plus hits a day? while at
> the same time cramming info from these hits into a
> database(berkley db)?
> 
> Ive got zope running on a mainframe and we are slowly
> approaching a point where one of our sites will be
> achieving such hits(less that 0.1kb per hit) ...
> 
> Im just curioso....
> 
> Wayne
> 
> __________________________________
> Do you Yahoo!?
> SBC Yahoo! DSL - Now only $29.95 per month!
> http://sbc.yahoo.com
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )