Hi, I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server. Now for the problem/questions. Problem: unacceptable performance with 200 requests/sec. Questions: - which steps can be taken to improve performance ? for the moment 6 child processes are running which each use 60 MB of memory (?isn't this alot) ; increasing the number of threads will probably help, but if each thread will probably use 60 MB - how good is the smp-support of Zope (Python) ? Thx in advance, ronny. __________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/
Linux doesn't report resource usage per thread. The number you see (60Mb) is shared between all threads, so if you have 5 threads then you are using 12Mb per thread. hth On Mon, 28 Oct 2002 03:24:37 -0800 (PST) John Doe <gvd_76@yahoo.com> wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
Now for the problem/questions. Problem: unacceptable performance with 200 requests/sec. Questions: - which steps can be taken to improve performance ? for the moment 6 child processes are running which each use 60 MB of memory (?isn't this alot) ; increasing the number of threads will probably help, but if each thread will probably use 60 MB - how good is the smp-support of Zope (Python) ?
Thx in advance, ronny.
__________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Take a look at ZEO and add some more machines to spread the load over multiple machines or multiple CPUs. -aj --On Montag, 28. Oktober 2002 03:24 -0800 John Doe <gvd_76@yahoo.com> wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
Now for the problem/questions. Problem: unacceptable performance with 200 requests/sec. Questions: - which steps can be taken to improve performance ? for the moment 6 child processes are running which each use 60 MB of memory (?isn't this alot) ; increasing the number of threads will probably help, but if each thread will probably use 60 MB - how good is the smp-support of Zope (Python) ?
Thx in advance, ronny.
__________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
--------------------------------------------------------------------- - Andreas Jung http://www.andreas-jung.com - - EMail: andreas at andreas-jung.com - - "Life is too short to (re)write parsers" - ---------------------------------------------------------------------
On Monday 28 October 2002 11:24 am, John Doe wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
Now for the problem/questions. Problem: unacceptable performance with 200 requests/sec. Questions: - which steps can be taken to improve performance ?
The answer probably involves caching. either inside Zope at application level, inside zope at ZODB level, or in front of Zope. You meed to measure where your bottleneck is, and put a cache in front of it. Another answer might involve a second machine, and ZEO. 200req/s isnt bad.
for the moment 6 child processes are running which each use 60 MB of memory (?isn't this alot) ;
No, thats 60M between them. This is not alot.
increasing the number of threads will probably help,
probably not if your machine is already cpu bound.
but if each thread will probably use 60 MB
zope memory usage is proportional to the number of threads, but that figure is a total of all threads. For this reason, you may (or may not) be better off with half as many threads each with a cache twice as big.
Hi, --On Montag, 28. Oktober 2002 03:24 -0800 John Doe <gvd_76@yahoo.com> wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
You can use ZEO even on the same machine. If you're running Zope and Postgres on the same machine you should either limit the nice level for postgres or move it to another machine, since postgres (and every other heavy process) can choke your zope instances a bit. But overall performance is influenced by many things, you schould carefully look at bottlenecks, improve sql statements, look where you can get many information at once with one request, cache results, cache even pages etc. Regards Tino
On Mon, 28 Oct 2002, Tino Wildenhain wrote:
You can use ZEO even on the same machine. If you're running Zope and Postgres on the same machine you should either limit the nice level for postgres or move it to another machine, since postgres (and every other heavy process) can choke your zope instances a bit. Is it possible to pin Zope and PostgreSQL to one prozessor in a dual processor machine? Just for the sake of interest ...
Kind regards Andreas.
What is the best way to identify bottlenecks ? Any products which provide this ? Is it possible to record the time used per page templates/script/... ? Cache: i'm already setting http-headers on pages to allow caching of user-independent pages; furthermore some pages are cached using a RAM Cache Manager --- Tino Wildenhain <tino@wildenhain.de> wrote:
Hi,
--On Montag, 28. Oktober 2002 03:24 -0800 John Doe <gvd_76@yahoo.com> wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
You can use ZEO even on the same machine. If you're running Zope and Postgres on the same machine you should either limit the nice level for postgres or move it to another machine, since postgres (and every other heavy process) can choke your zope instances a bit.
But overall performance is influenced by many things, you schould carefully look at bottlenecks, improve sql statements, look where you can get many information at once with one request, cache results, cache even pages etc.
Regards Tino
__________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/
Have you tried to enable "Big M" logging to get an idea if there are any typical requests taking a long time? -aj --On Montag, 28. Oktober 2002 04:59 -0800 John Doe <gvd_76@yahoo.com> wrote:
What is the best way to identify bottlenecks ? Any products which provide this ? Is it possible to record the time used per page templates/script/... ?
Cache: i'm already setting http-headers on pages to allow caching of user-independent pages; furthermore some pages are cached using a RAM Cache Manager
--- Tino Wildenhain <tino@wildenhain.de> wrote:
Hi,
--On Montag, 28. Oktober 2002 03:24 -0800 John Doe <gvd_76@yahoo.com> wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
You can use ZEO even on the same machine. If you're running Zope and Postgres on the same machine you should either limit the nice level for postgres or move it to another machine, since postgres (and every other heavy process) can choke your zope instances a bit.
But overall performance is influenced by many things, you schould carefully look at bottlenecks, improve sql statements, look where you can get many information at once with one request, cache results, cache even pages etc.
Regards Tino
__________________________________________________ Do you Yahoo!? Y! Web Hosting - Let the expert host your web site http://webhosting.yahoo.com/
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
--------------------------------------------------------------------- - Andreas Jung http://www.andreas-jung.com - - EMail: andreas at andreas-jung.com - - "Life is too short to (re)write parsers" - ---------------------------------------------------------------------
On Mon, 28 Oct 2002 11:59 pm, John Doe wrote:
What is the best way to identify bottlenecks ? Any products which provide this ? Is it possible to record the time used per page templates/script/... ?
Have you looked at this yet? http://www.zope.org/Members/richard/docs/zope_optimisation.html CallProfiler, from: http://www.zope.org/Members/richard/CallProfiler was supposed to be integrated into Zope 2.6 but missed the boat. It works with 2.5 and 2.6 with a little tweaking. I hope to release new versions soon. Richard
and for python products there is of course also the standard profiler (i think CallProfiler doesn't work with python products but i may be mistaken), to be found in /Control_Panel/DebugInfo/manage_profile. for instance with the profiler i recently found that my only performance problem was the ZODBC adapter resp. ODBC :-) - peter. Richard Jones wrote:
On Mon, 28 Oct 2002 11:59 pm, John Doe wrote:
What is the best way to identify bottlenecks ? Any products which provide this ? Is it possible to record the time used per page templates/script/... ?
Have you looked at this yet?
http://www.zope.org/Members/richard/docs/zope_optimisation.html
CallProfiler, from:
http://www.zope.org/Members/richard/CallProfiler
was supposed to be integrated into Zope 2.6 but missed the boat. It works with 2.5 and 2.6 with a little tweaking. I hope to release new versions soon.
Richard
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Tue, 29 Oct 2002 10:02 am, peter sabaini wrote:
and for python products there is of course also the standard profiler (i think CallProfiler doesn't work with python products but i may be mistaken), to be found in /Control_Panel/DebugInfo/manage_profile.
for instance with the profiler i recently found that my only performance problem was the ZODBC adapter resp. ODBC :-)
You can add arbitrary methods to the CallProfiler list. The major benefit with the CallProfiler is the results analysis. The regular python profiler can be a little difficult to decipher - mostly because it profiles _everything_. Richard
John Doe wrote:
Hi,
I'm running a Zope-application (2.5.1, Python 2.1.3) on a dual processor (1 GB mem) Redhat 7.3 machine. User-authentication is done by the exUserFolder-product (on Postgresql 7.2.1). An apache-server runs in front of the Zope server.
Now for the problem/questions. Problem: unacceptable performance with 200 requests/sec. Questions: - which steps can be taken to improve performance ? for the moment 6 child processes are running which each use 60 MB of memory (?isn't this alot) ; increasing the number of threads will probably help, but if each thread will probably use 60 MB - how good is the smp-support of Zope (Python) ?
Keep in mind that Python uses a global interpreter lock (GIL) to serialize access to the Python interpreter between threads. In multiprocessor environments, this can degrade performance when the operating system schedules work for CPUs which do not hold the GIL. I would expect Zope's rendering performance on a uniprocessor AMD Athlon 2000+ to be on the order of 15-30 pages per second for a reasonably complex site. "Complexity" is closely correlated to the amount of objects needed to be accessed by the interprester -- the more things your pages include, the slower they will render. The key to high volume sites is to render a page once, then cache it. You can use Zope's caching, or an external cache such as squid. Zope Corporation uses squid to accelerate sites they host. Zope's performance scales linearly with the pystone benchmark. The "fixed cost" of asking Zope to serve something is much higher than the fixed cost of asking apache or squid to serve something -- Zope can still serve hundreds of images per second on a fast machine, but squid and apache can serve thousands. Zope is a reasonably complex piece of software, and it's not easy to do much more than give "default" pointers for tuning. One thing you can do is raise your ZODB cache size in the control panel. You should probably run multiple Zope instances, using a ZEO storage. You should use apache or squid in front of Zope to prevent Zope from rendering the same page multiple times. -- Matt Kromer Zope Corporation http://www.zope.com/
participants (10)
-
Andreas Jung -
Andreas Tille -
Dieter Maurer -
John Doe -
Matthew T. Kromer -
peter sabaini -
Phil Harris -
Richard Jones -
Tino Wildenhain -
Toby Dickenson