Can this be taken to mean that: - the practical maximum number of threads to run your single (non-ZEO) zope instance is {number of zodb connections in pool} else you risk deadlock if yes, would upping the number of ZODB connections effectively raise the ceiling - e.g. 12 ZODB connections -> 12 threads should perform properly ? Is increasing the number of ZODB cx's possible, let alone advisable? (why the default of 7 - not 6 or 8?) More concretely, is there any good way to increase the request-handling capacity of a standalone instance, beyond the limits imposed by the defaults, without deploying ZEO?
To put it in a very simple way, you should not tweak the threads setting unless you really know what you are doing. If your goal is performance the usual way of going about that is to run ZEO and add ZEO clients as needed.
You're bumping into the very simple fact that there are two items interacting badly: Threads and the connections in the Zope database connection pool. A thread, under normal circumstances, can only do work if it has a connection to work with, and wwill only release the connection when the request is finished. There are 7 connections in the pool by default.
jens
Can this be taken to mean that:
- the practical maximum number of threads to run your single (non-ZEO) zope instance is {number of zodb connections in pool} else you risk deadlock
No, there should be more connections than threads, just as in the original configuration.
if yes, would upping the number of ZODB connections effectively raise the ceiling - e.g. 12 ZODB connections -> 12 threads should perform properly ? Is increasing the number of ZODB cx's possible, let alone advisable? (why the default of 7 - not 6 or 8?)
More concretely, is there any good way to increase the request-handling capacity of a standalone instance, beyond the limits imposed by the defaults, without deploying ZEO?
Trying to squeeze a single instance that way is fraught with problems, and not just the experimentation needed to come up with sane values for threads, connections, and then connection cache sized to avoid memory contention. A single Zope process also means you are running one Python process with one global interpreter lock. It's just a bad situation to base a production site on. Use ZEO and set up additional clients as needed, anything else is hackish and error-prone. jens
On 29. Apr 2005, at 15:34, Jim Abramson wrote:
Can this be taken to mean that:
- the practical maximum number of threads to run your single (non-ZEO) zope instance is {number of zodb connections in pool} else you risk deadlock
You will want to make the connection pool a bit larger than the number of threads you start. But see below.
if yes, would upping the number of ZODB connections effectively raise the ceiling - e.g. 12 ZODB connections -> 12 threads should perform properly ? Is increasing the number of ZODB cx's possible, let alone advisable? (why the default of 7 - not 6 or 8?)
Upping the number of threads is unlikely to give you better performance. The only case where this could make sense is if you had something like a highly saturated RDBMS backend, tying up your worker threads. But - Zope threads don't operate like you probably expect from knowing Apache and similar models. For one, they *never* will run in parallel. Python employs a global interpreter lock (GIL), so there will only be a single thread "working" at all times. What you want is to up the number of processes (interpreters) not the number of threads. Hence ZEO.
More concretely, is there any good way to increase the request-handling capacity of a standalone instance, beyond the limits imposed by the defaults, without deploying ZEO?
Caching is essential. Put up a squid. And buy real hardware. Zope won't give you love on PC-class devices. Buy more RAM, and then some more. And there is nothing wrong with ZEO, really, there's only advantages. I run it on my laptop even. On modern hardware you can easily run 2-4 ZEO clients per CPU. HTH, Stefan -- Software Engineering is Programming when you can't. --E. W. Dijkstra
On Sun, May 01, 2005 at 01:34:08PM +0200, Stefan H. Holek wrote:
And there is nothing wrong with ZEO, really, there's only advantages.
One potentially large quibble: If you have big (multi-megabyte) blobs in your ZODB, and this data is not cached in the client's ZEO cache, performance is an order of magnitude worse when running ZEO. Once the data is in the client cache, performance is about the same as a standalone zope running filestorage. One workaround is to set the client cache size very large. -- Paul Winkler http://www.slinkp.com
participants (4)
-
Jens Vagelpohl -
Jim Abramson -
Paul Winkler -
Stefan H. Holek