[ZODB-Dev] Very Large Application Use Case

[Joachim Werner]

>>I have not tried using Zope in a scenario that large yet. And I am
>>pretty sure that it would take a lot of hardware to serve that many
>>users. But I can estimate by multiplying the numbers I know of.
>>
>>Here are the facts I'd need:
>>
>>- What will be the read/write ratio?
> 
> 
> I would estimate a 60 - 40 split.

That's a lot of writing ...


>>- Will the content be static or dynamic?

> 95% dynamic

So no caching ...

> Partitioning the app may help things, however one of the design goals it to 
> have a unified system.  Something like NDS and multiple Netware servers may 
> work.

>>- How "real-time" is the application?
> 
> 
> It isn't a "real-time" application, but needs to be able to make data 
> available within a maximum of  10 - 20 seconds.

That's as real-time as a web application can get I guess ;-)

The numbers I have suggest that you will need a lot of hardware:

Pentium 3, 800 MHz (11000 pystones/s):

- simple Zope page (return the REQUEST variable called via DTML):
   66 pages/s

- CMS homepage (no data from ZCatalog): 5-10 pages/s

Pentium IV ? MHz (20.000 pystones/s):

- simple Zope page (return the REQUEST variable called via DTML):
   100 pages/s

- CMS homepage (news and events from ZCatalog, more automatically 
generated navigation elements than in the first example: 0,5-1 pages/s

You need an average of 222 transactions (whatever those will be exactly) 
per second, which means that you'd need a top performance of 400 
transactions/s or more.

If your transactions are comparable with getting the CMS homepages,
you'll end up needing between 40 and 800 servers. That's almost Google ...

With this number of servers you will probably have scalability issues in 
the ZEO architecture (sending cache invalidation messages to the clients 
will take time).

There is another number to compare with:

http://www.sap-ag.de/benchmark/index.asp?content=http://www.sap-ag.de/benchmark/sd2tier.asp

This is the SAP application benchmark. It benchmarks one of the the SAP 
R/3 enterprise resource management software modules. This software is, 
basically, a large SQL-based client-server application.

To get 800.000 transactions per hour -- they call them "dialog steps" -- 
(one dialog step invokes a number of SQL queries), you will need a 
32-processor Itanium2 or Alpha server. They have less than 3000 
concurrent users, but a user does a lot more transactions than in your 
scenario.

The SAP benchmark is probably completely irrelevant to your scenario, 
but so is getting numbers from a totally different Zope application.

If you want to get this thing done with less than one or two racks full 
of blade servers you will have to write very optimized code. Zope will 
definitely not be the best solution for the core engine, but it might be 
a very good solution for serving the actual web sites.

I guess that's all I can contribute.

Joachim







-- 

iuveno AG

Joachim Werner

_________________

Wittelsbacherstr. 23b
90475 Nürnberg

joachim.werner@iuveno.de
www.iuveno.de

Tel.: +49 (0) 911/ 9 88 39 84