Can Zope handle 10 billion plus hits a day? while at the same time cramming info from these hits into a database(berkley db)? Ive got zope running on a mainframe and we are slowly approaching a point where one of our sites will be achieving such hits(less that 0.1kb per hit) ... Im just curioso.... Wayne __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com
Maybe. That's a lot of traffic. Ten billion hits per day equates to 115,740 hits per second of continuous steady-state traffic (10,000,000,000 / 86400). A typical rule of thumb for system scaling is that 5 * nonpeak hits/sec needs to be accounted for in the design, so you'd probably need to design for about 580,000 hits/sec to meet peak load. If you can offload 90% of the traffic to a caching tier (very possible if you don't need to do many writes), that means every 1 out of 10 requests or so will make it to a Zope appserver. You can also assume that a single cache server on Intel hardware (you'll need to convert this to mainframe terms) can handle about 600 requests per second. Thus, your application will require about 966 cache server instances (580,000 / 600). If 10% of the traffic makes it to Zope, this means that the Zope tier will need to service about 58,000 requests per second. If you design your application properly, you can expect a hit rate from a single Zope instance on Intel hardware of about 50 - 90 requests/sec. This means you're going to need at least 644 appservers (58,000 *.1 / 90). I don't know how many ZEO servers this setup would need, we've never needed to project ZEO server scale like that. Probably about 50. FWIW, just to get an idea of the scale of something to handle this: - Your estimated bandwidth requirements for such a setup will be about 1160 Mbits/s during peak load (if you assume each hit consumes about 250 bytes to account for overhead, which is a very low estimate). - If a T1 costs US $700 a month, you'll be spending ~$525,000.00 a month (1160 / 1.544 * 700) retail just for the connectivity charges to meet your peak demand. - The initial hardware investment (if you need it) will cost about US$3.5 million+ (1660 servers * 2100/apiece). - If you figure that one competent sysadmin can administer about 60 machines, you'll likely need to hire around 26 full time admins to keep the system going. Given all that, are you sure about your hit projection? - C On Mon, 2003-07-07 at 21:40, Wayne Connolly wrote:
Can Zope handle 10 billion plus hits a day? while at the same time cramming info from these hits into a database(berkley db)?
Ive got zope running on a mainframe and we are slowly approaching a point where one of our sites will be achieving such hits(less that 0.1kb per hit) ...
Im just curioso....
Wayne
__________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On Mon, Jul 07, 2003 at 11:39:56PM -0400, Chris McDonough wrote:
- If a T1 costs US $700 a month, you'll be spending ~$525,000.00 a month (1160 / 1.544 * 700) retail just for the connectivity charges to meet your peak demand.
That's not a fair number -- scaling up to 1160Mbits doesn't mean you'll have 700 T1s (my God, imagine the number of routers you'd use -- one of the denser platforms I've seen is the Cisco 7206VXR, and even that would only do about 40 T1s). You'd go for something like OC-24 and load-balance it over two GigE links.
- The initial hardware investment (if you need it) will cost about US$3.5 million+ (1660 servers * 2100/apiece).
FWIW, that's not necessarily all that much (though you haven't factored in support costs and networking hardware costs). I've seen infrastructure builds at sites that featured SIGNIFICANTLY less traffic that cost that much. -roy
On Mon, 2003-07-07 at 23:46, Roy S. Rapoport wrote:
On Mon, Jul 07, 2003 at 11:39:56PM -0400, Chris McDonough wrote:
- If a T1 costs US $700 a month, you'll be spending ~$525,000.00 a month (1160 / 1.544 * 700) retail just for the connectivity charges to meet your peak demand.
That's not a fair number -- scaling up to 1160Mbits doesn't mean you'll have 700 T1s (my God, imagine the number of routers you'd use -- one of the denser platforms I've seen is the Cisco 7206VXR, and even that would only do about 40 T1s). You'd go for something like OC-24 and load-balance it over two GigE links.
Sure. I was trying to drive home the scale of such an effort by ballparking the cost using numbers I know and I haven't shopped around for OC-24 links lately. ;-)
- The initial hardware investment (if you need it) will cost about US$3.5 million+ (1660 servers * 2100/apiece).
FWIW, that's not necessarily all that much (though you haven't factored in support costs and networking hardware costs). I've seen infrastructure builds at sites that featured SIGNIFICANTLY less traffic that cost that much.
Well, you heard him Wayne, don't consider my quote binding. ;-) - C
participants (3)
-
Chris McDonough -
Roy S. Rapoport -
Wayne Connolly