[Zope] Defining High Load in Heavily DB Driven site (was: [Zope] zope2.2.0 and what is high load)

[Paul Everitt]

"Dario Lopez-Kästen" wrote:
> 
> Hello!

Hello Dario!

> I need to find more info on how Zope handles threading with relation to
> database connections.

How about:

  http://www.zope.org/Documentation/Developer/Models/ZODB

The jargon of threads and connections (and then threads again) can be
pretty confusing in Zope.

First, ZServer is the thing in front listening for socket requests.  It
has a connection pool (select based) which maps into Python threads in
the Zope application.  (ZServer is part of the Zope process, I'm
discussing logical partitioning.)

Each concept of a "thread" in Zope is actually a "database connection"
to the ZODB.  Threads get a private copy of the transaction space, thus
greatly simplifying (to the point of removal) the need for programmers
to think about thread safety.

If you're talking about the thread concurrency a connection to Oracle,
then that's a bit different.  I _think_ that ultimately this maps to the
number of Zope database connections.

This conversation is probably better in the zope-dev list.

> I am currently considering and evaluating Zope as one of the options we have
> to build a really large, completely databasedriven "enterprise scale"
> web-platform. I am a bit worried about this "maximum of 7 threads per db

This isn't a maximum.  It's just the default.  It's easy to increase the
number of connections.

> connection" limitation mentioned previously, and I need to know a) what does
> this mean in terms of accesibility, b) how does it affect performace, and c)
> how does Zope work (in detail) with external database connections.
> 
> We are going to be using Oracle as our DB backend, all our served data will
> be database *only* (some of it will be fairly large), and efficient database
> connections is *crucial* to what we intend to do.

Since you're not using any persistent objects in Zope (unless you do
caching of the SQL data, which you probably should), then you don't have
to worry about process growth from having more database connections for
Zope.  Jack it up to 25.

Since much of the time in a Zope hit is taken by rendering to HTML, a
number like 25 probably means you can process a hundred simultaneous
Oracle requests.  That works out to be an average of around ten million
hits per day.  If you want more, increase the number.

> We are going to have between 10-20k users at most and I expect the maximun
> of simultaneous connections to be beween 400 - 900 under heavy load; "normal
> load" will probably oscillate around 25-200 simultaneous connections (these
> numbers are crude guesses, but take into account increased usage over time
> as more services are provided in the future).
> 
> Bandwith problems are not an issue, nor is computing power.
> 
> I would appreciate any pointers to documentation or people to ask.

Is this an intranet or a public site?

--Paul