Some time ago, there was a brief thread about zope on Sun. Does Python 1.5.2 run slower than might be hoped ? Two examples - taken from real boxes running real jobs. The bigger box was running more/other tasks than the littler one but load average never got beyond 1.0. Note that the Lintel box was a midvintage PII350 (92mB ram !) server running a squid proxy - surely not twice as fast as a lightly loaded dual cpu sun ?. Exhibit one: pystone.py on sin - Pystone(1.1) time for 10000 passes = 5.57 This machine benchmarks at 1795.33 pystones/second On the LIntel box it was: Pystone(1.1) time for 10000 passes = 2.36 This machine benchmarks at 4237.29 pystones/second Exhibit 2: A self timing site generating (zclass based - no ZSQL) zope script on a Linuxtel box with Python 1.5.2 (#1, Apr 14 1999, 10:34:26) [GCC 2.7.2.3] on linux2 took 1min:10sec. On a dual cpu Sun called sin, (running Python 1.5.2 (#1, Oct 13 1999, 11:11:12) [GCC 2.95.1 19990816 (release)]) on sun os5, it took 2min:48sec. Is it GCC that makes the sun slower than one might hope ? Anyone ? -- Ross Lazarus Email: rossl@med.usyd.edu.au http://www.health.usyd.edu.au/people/rossl.htm
Some time ago, there was a brief thread about zope on Sun. Does Python 1.5.2 run slower than might be hoped ?
Evidence seems to point that way. Our Enterprise 450 gave this. Zope 2.0.0 binary distribution: Python 1.5.2 (#2, Jul 9 1999, 16:07:49) [GCC 2.8.1] on sunos5 Pystone(1.1) time for 10000 passes = 3.93 This machine benchmarks at 2544.53 pystones/second Python compiled by me: Python 1.5.2 (#5, Nov 5 1999, 11:08:13) [GCC 2.95.1 19990816 (release)] on sunos5 Pystone(1.1) time for 10000 passes = 3.42 This machine benchmarks at 2923.98 pystones/second Another solaris box (smaller than the E450) using egcs: Python 1.5.2 (#2, Sep 15 1999, 10:11:04) [GCC egcs-2.91.66 19990314 (egcs-1.1.2 on sunos5 Pystone(1.1) time for 10000 passes = 3.47 This machine benchmarks at 2881.84 pystones/second So there may be something in
Exhibit one: pystone.py on sun -
Pystone(1.1) time for 10000 passes = 5.57 This machine benchmarks at 1795.33 pystones/second
On the LIntel box it was:
Pystone(1.1) time for 10000 passes = 2.36 This machine benchmarks at 4237.29 pystones/second
[snip]
Is it GCC that makes the sun slower than one might hope ? Anyone ?
It may be, but I'm not really qualified to say...as I've only just got a compiled Python working on our E450 and am loathe to try out different compilers! :) Tone ps my iMac 266MHz scored 4500 pystones..maybe I should take my Mac into work and bring the E450 home?
FWIW, some other pystone benchmarks: Cobalt Qube: 432 Cobalt Qube2: 1081 Celeron 400: 5405 I wonder what the G4s would look like?
Evidence seems to point that way. Our Enterprise 450 gave this. Zope 2.0.0 binary distribution: Python 1.5.2 (#2, Jul 9 1999, 16:07:49) [GCC 2.8.1] on sunos5 Pystone(1.1) time for 10000 passes = 3.93 This machine benchmarks at 2544.53 pystones/second
Another solaris box (smaller than the E450) using egcs: Python 1.5.2 (#2, Sep 15 1999, 10:11:04) [GCC egcs-2.91.66 19990314 (egcs-1.1.2 on sunos5 Pystone(1.1) time for 10000 passes = 3.47 This machine benchmarks at 2881.84 pystones/second
ps my iMac 266MHz scored 4500 pystones..maybe I should take my Mac into work and bring the E450 home?
On Sat, 6 Nov 1999 Tony.McDonald@newcastle.ac.uk wrote:
Evidence seems to point that way. Our Enterprise 450 gave this. Zope 2.0.0 binary distribution: Python 1.5.2 (#2, Jul 9 1999, 16:07:49) [GCC 2.8.1] on sunos5 Pystone(1.1) time for 10000 passes = 3.93 This machine benchmarks at 2544.53 pystones/second
And on an UltrasparcII 200Mhz, using Sun's CC -O: Pystone(1.1) time for 10000 passes = 3.9 This machine benchmarks at 2564.1 pystones/second What speed are your CPU's running at? Ultrasparcs are available up to 450MHz (or is it 500 now?) so pystones should be around the 5700 mark (I'll be able to test when our new Oracle server arrives...). The benchmarks in this email are on an UltrasparcII 200Mhz - I don't think Sun sell CPU's this slow any more.
It may be, but I'm not really qualified to say...as I've only just got a compiled Python working on our E450 and am loathe to try out different compilers! :)
Using Sun cc with almost all the optimizations on (-fast -mt -xarch=v9 -xcode=pic32 -xO5 -xprefetch -xsafe=mem), I get with the same UltraII 200MHz: Pystone(1.1) time for 10000 passes = 3.7 This machine benchmarks at 2702.7 pystones/second Using 'gcc version egcs-2.91.66 19990314 (egcs-1.1.2 release)' on the same box I get: Pystone(1.1) time for 10000 passes = 4.28 This machine benchmarks at 2336.45 pystones/second ___ // Zen (alias Stuart Bishop) Work: zen@cs.rmit.edu.au // E N Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Two examples - taken from real boxes running real jobs. The bigger box was running more/other tasks than the littler one but load average never got beyond 1.0. Note that the Lintel box was a midvintage PII350 (92mB ram !) server running a squid proxy - surely not twice as fast as a lightly loaded dual cpu sun ?.
If your only benchmark is CPU grunt, in Australia an UltraSparcII will cost 50-100% more than the equivalent Intel based box (assuming quality namebrand hardware, same support levels & warrenty etc).
Pystone(1.1) time for 10000 passes = 5.57 This machine benchmarks at 1795.33 pystones/second
On the LIntel box it was:
Pystone(1.1) time for 10000 passes = 2.36 This machine benchmarks at 4237.29 pystones/second
Pystones seems to be a single threaded CPU benchmark. For raw CPU grunt, a PII350 will outperform an UltraII 250, and possibly an UltraII 450. You didn't mention what speed the sun's CPU's are running, and what architecture (sparc, ultrasparc, ultrasparcII). Squid won't effect the pystones reading much - it generally doesn't need much cpu time.
Exhibit 2: A self timing site generating (zclass based - no ZSQL) zope script on a Linuxtel box with Python 1.5.2 (#1, Apr 14 1999, 10:34:26) [GCC 2.7.2.3] on linux2 took 1min:10sec.
On a dual cpu Sun called sin, (running Python 1.5.2 (#1, Oct 13 1999, 11:11:12) [GCC 2.95.1 19990816 (release)]) on sun os5, it took 2min:48sec.
Try running four or five of these scripts simultaneously and see what happens. You will then see if the second CPU in the sun helps significantly. You may be correct about gcc on sparc being slow - I believe that much more of gcc's tuning has been done with i386 code rather than sparc code. We build entirely with sun's cc and CC here. On this note, has anyone got any canned Zope 'excercise' programs? I need to do some stress tests in various configurations (raw zserver, via apache, via apache ssl, with zodb on a nfs mount from the NetApp, with zodb on local disk, performance improvements with memory increase) ___ // Zen (alias Stuart Bishop) Work: zen@cs.rmit.edu.au // E N Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen
On Sun, Nov 07, 1999 at 10:23:37AM +1100, Stuart 'Zen' Bishop wrote:
On this note, has anyone got any canned Zope 'excercise' programs? I need to do some stress tests in various configurations (raw zserver, via apache, via apache ssl, with zodb on a nfs mount from the NetApp, with zodb on local disk, performance improvements with memory increase)
I use ApacheBench (ab) from the Apache distribution. -Petru
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Some time ago, there was a brief thread about zope on Sun. Does Python 1.5.2 run slower than might be hoped ?
Two examples - taken from real boxes running real jobs. The bigger box was running more/other tasks than the littler one but load average never got beyond 1.0. Note that the Lintel box was a midvintage PII350 (92mB ram !) server running a squid proxy - surely not twice as fast as a lightly loaded dual cpu sun ?. You should perhaps run the pystone benchmarks with time around it, and divide by the CPU percentage. This should correct at least partly for some existing load. (It doesn't solve the problem that with heavier load the caches tend to be more often cold, etc.)
Andreas -- Andreas Kostyrka | andreas@mtg.co.at phone: +43/1/7070750 | phone: +43/676/4091256 MTG Handelsges.m.b.H. | fax: +43/1/7065299 Raiffeisenstr. 16/9 | 2320 Zwoelfaxing AUSTRIA
Discussing pystone benchmarks ... On our server Dual 450 PentiumIII, 512 Mb Ram, running ZServer (30,000 hits /per day) and apache (800,000 req/day) pystone gives: Pystone(1.1) time for 10000 passes = 1.9 This machine benchmarks at 5263.16 pystones/second Comparing with other machines I got the impression the pystones does not do justice for dual processor machines. Regards Pavlos On Sun, 7 Nov 1999, Andreas Kostyrka wrote:
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Some time ago, there was a brief thread about zope on Sun. Does Python 1.5.2 run slower than might be hoped ?
Two examples - taken from real boxes running real jobs. The bigger box was running more/other tasks than the littler one but load average never got beyond 1.0. Note that the Lintel box was a midvintage PII350 (92mB ram !) server running a squid proxy - surely not twice as fast as a lightly loaded dual cpu sun ?. You should perhaps run the pystone benchmarks with time around it, and divide by the CPU percentage. This should correct at least partly for some existing load. (It doesn't solve the problem that with heavier load the caches tend to be more often cold, etc.)
Andreas -- Andreas Kostyrka | andreas@mtg.co.at phone: +43/1/7070750 | phone: +43/676/4091256 MTG Handelsges.m.b.H. | fax: +43/1/7065299 Raiffeisenstr. 16/9 | 2320 Zwoelfaxing AUSTRIA
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope No cross posts or HTML encoding! (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Benchmarks are always problematic. It seems pretty clear that pystone is single threaded and doesn't do "justice" to dual cpu's. It's also clear that any serious benchmarks should run multiple threads to compare machines under load when running zope tasks. But, I still have trouble understanding why a single zope task takes twice as long on the lightly loaded midrange Sun as a low end LIntel box ! It's a typical development task - make 70 complex pages from zclasses with thousands of elements. I don't want it to run multithread ! Coming from a PC background, I always thought that we bought "big iron" minis because they gave better throughput for everything - sure, it's not that simple and the big iron will no doubt do much better under heavier load.... FWIW, I'm moving all my development to the $2k LIntel box until I get faster (raw, single threaded !) response elsewhere.... Pavlos Christoforou wrote:
Discussing pystone benchmarks ...
On our server Dual 450 PentiumIII, 512 Mb Ram, running ZServer (30,000 hits /per day) and apache (800,000 req/day) pystone gives:
Pystone(1.1) time for 10000 passes = 1.9 This machine benchmarks at 5263.16 pystones/second
Comparing with other machines I got the impression the pystones does not do justice for dual processor machines.
Regards
Pavlos
On Sun, 7 Nov 1999, Andreas Kostyrka wrote:
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Some time ago, there was a brief thread about zope on Sun. Does Python 1.5.2 run slower than might be hoped ?
Two examples - taken from real boxes running real jobs. The bigger box was running more/other tasks than the littler one but load average never got beyond 1.0. Note that the Lintel box was a midvintage PII350 (92mB ram !) server running a squid proxy - surely not twice as fast as a lightly loaded dual cpu sun ?. You should perhaps run the pystone benchmarks with time around it, and divide by the CPU percentage. This should correct at least partly for some existing load. (It doesn't solve the problem that with heavier load the caches tend to be more often cold, etc.)
Andreas -- Andreas Kostyrka | andreas@mtg.co.at phone: +43/1/7070750 | phone: +43/676/4091256 MTG Handelsges.m.b.H. | fax: +43/1/7065299 Raiffeisenstr. 16/9 | 2320 Zwoelfaxing AUSTRIA
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope No cross posts or HTML encoding! (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- Dr Ross Lazarus Associate Professor and Sub-Dean for Information Technology Faculty of Medicine, A27, R126a, University of Sydney, Camperdown, NSW 2006, Australia Tel: (+61 2) 93514429 Mobile: +61414872482 Fax: (+61 2) 93516646 Email: rossl@med.usyd.edu.au http://www.health.usyd.edu.au/people/rossl.htm
On Sun, Nov 07, 1999 at 01:44:02PM +1100, Dr. Ross Lazarus wrote:
Benchmarks are always problematic. It seems pretty clear that pystone is single threaded and doesn't do "justice" to dual cpu's. It's also clear that any serious benchmarks should run multiple threads to compare machines under load when running zope tasks.
But, I still have trouble understanding why a single zope task takes twice as long on the lightly loaded midrange Sun as a low end LIntel box !
It's a typical development task - make 70 complex pages from zclasses with thousands of elements. I don't want it to run multithread ! Coming from a PC background, I always thought that we bought "big iron" minis because they gave better throughput for everything - sure, it's not that simple and the big iron will no doubt do much better under heavier load.... Well, the problem is, that the big iron excells mostly on: -) IO bandwidth. -) High avail. stuff. (Sometimes going as far as hotplugging CPUs ;) ) -) heavy load. -) Scaling.
Basically, while you might see better benchmarks with a PC and EIDE drives, when you put the pressure of multiuser usage on it, the PC quickly breaks, while the big iron nicely chucks along. On the anectodal level: You can even see it with PCs, where I only once tried to replace old SCSI drives with a new (and at least 3 times faster in all categories) EIDE drive in a multiuser (X Terminal users via xdm) server. The box was faster with one user. With two users it was bad, and with three users the interactive performance was inacceptable. Well after one week, the box did have SCSI drives again ;)
FWIW, I'm moving all my development to the $2k LIntel box until I get faster (raw, single threaded !) response elsewhere....
Well, the problem here is, that you should try to benchmark some easy page which with Zope is almost never completly static (standard_html_header/footer) with ab (Apache benchmark), and let do multiple requests at the same time. That is a more practical Zope benchmark then pystone. ;) Andreas -- Andreas Kostyrka | andreas@mtg.co.at phone: +43/1/7070750 | phone: +43/676/4091256 MTG Handelsges.m.b.H. | fax: +43/1/7065299 Raiffeisenstr. 16/9 | 2320 Zwoelfaxing AUSTRIA
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Is it GCC that makes the sun slower than one might hope ? In theory this could quite be so. gcc always tended to be worse for RISC architectures, and egcs could have substantly improved on the gcc optimizations ;)
Andreas -- Andreas Kostyrka | andreas@mtg.co.at phone: +43/1/7070750 | phone: +43/676/4091256 MTG Handelsges.m.b.H. | fax: +43/1/7065299 Raiffeisenstr. 16/9 | 2320 Zwoelfaxing AUSTRIA
On Sun, 07 Nov 1999, Andreas Kostyrka wrote:
On Sat, 6 Nov 1999, Dr. Ross Lazarus wrote:
Is it GCC that makes the sun slower than one might hope ? In theory this could quite be so. gcc always tended to be worse for RISC architectures, and egcs could have substantly improved on the gcc optimizations ;)
While this is not really relivent here.. GCC tends to be MUCH better on risc optimisations than on intel, at least historically. One of the primary reasons for the EGCS spilt was to add pentium optimisations (a LONG time after the pentium was out), as the gcc maintainers were only really showing interest in risc type updates, and ignoring the intel family, which is only just starting to catch up in the latest gccs (now that egcs has turned back into gcc...) Then again, gcc is not great compared to most compilers produced by the primary chip supporter for just about all architectures, including sparc, alpha, intel pertium family, etc.. sometimes it gets close, and it tends (on average) to be MUCH more standard these days. you will find that linux on the sun boxes will execute this type of code faster than solaris will, as it has a much lower system calling overhead (if the boxes in question are of the few suns which can run linux..) -- ------------------------------------------------------------ Stuart Woolford, stuartw@newmail.net Unix Consultant. Software Developer. Supra Club of New Zealand. ------------------------------------------------------------
participants (9)
-
Andreas Kostyrka -
Andreas Kostyrka -
Dr. Ross Lazarus -
Luke Tymowski -
Pavlos Christoforou -
Petru Paler -
Stuart 'Zen' Bishop -
Stuart Woolford -
Tony.McDonald＠newcastle.ac.uk