Well... it looks like that Sun guy wasn't lying about performance with Forte... I rebuilt Python 2.1.1 from source and ran several passes of Pystone. With gcc, Pystone averages about 4000 pystones/sec. With Forte, it's 4300! Consistently. That's a good 7-8% increase in performance. As for threads... both gcc and Forte compiled python seems to do the same thing with threads (like you described... it creates all threads, then waits for completion). I've attached the output of both tests. GCC python was compiled with the following options: ./configure --with-threads Forte python compiled with: ./configure --without-gcc -with-cxx=CC --with-threads --prefix=/opt/python Both python executables were 32-bit with no Ultrasparc optimizations... I'm using the latest version of Forte - v6 update 2. For fun, I was going to try building a 64-bit UltraSparc II optimized python (didn't work, I'm still trying though), but I did find in the Python source distro README, that using Sun Workshop compilers you should add the -Xc -mt CFLAGS. -mt for "multi-threading". I then went through and built another python like this.... Pystone benchmarks are now 4360-4380. That's an almost 10% boost over straight gcc (no change in the threads test). Looks like gcc has some work to do on Sparc. I'd be happy to distribute the compiled versions of Python I have... What's a good way to do this for maximum portability? cheers, Paul. At 08:45 AM 12/1/2001 +0000, tone.mcdonald@ncl.ac.uk wrote:
On Friday, November 30, 2001, at 08:13 PM, Paul Horbal wrote:
Has anybody attempted to do this?
I'm finding that even though I specify CC, CXX, put the Forte directories ahead of /usr/local/bin in my path, Python is still falling back on gcc to build Zope. Where can I modify this behaviour?
Our Sun rep tells us that we can kiss performance goodbye using gcc... Since we have Forte anyways, I'd like to try it out.
thanks, Paul.
Good luck Paul - there are some issues with python under sun hw and/or solaris, which inevitably lead to issues with Zope.
#1: speed: the pystone benchmark on our enterprise 3500 (400 MHz Ultrasparc) is about 4500 (using gcc compiler). My TiPB using MacOS-X gets 6500 and a PIII/700MHz gets over 10,000. This has lead to me trying to use ZEO on our system (which has multiple processors), but I ran up against a problem using CoreSessionTracking needing a mountable storage that I couldn't get my head around.
Don't get me wrong - all our Zope systems work (although I'm getting python2.1 core dumps which are *maddening*).
#2: threading: Zope is a multi-threaded beast and this allows it to handle more than one request at a time. This does not seem to work under Solaris. Check the archives for "threading, dtml and performance". Here's a URL from NIPltd http://zope.nipltd.com/public/lists/zope- archive.nsf/47ba74c812dbc5dd8025687f0024bb5f/ebe6af5e2cd7b04780256af1002b7440? OpenDocument&Highlight=0,threading and one from Activestate http://aspn.activestate.com/ASPN/search?query=threading&type=Archive_zope-Li... Try out the script that Oliver Erlewein posted and see what happens on your own Solaris box.
What seems to indicate 'good' threading is if you get this kind of output from
155[8:22]tonymcd@localhost ~ % python /usr/local/lib/python2.1/test/test_thread.py creating task 1 task 1 will run for 1.9 sec creating task 2 task 2 will run for 4.6 sec creating task 3 task 3 will run for 3.0 sec creating task 4 task 4 will run for 2.6 sec creating task 5 task 5 will run for 8.8 sec creating task 6 task 6 will run for 9.3 sec creating task 7 task 7 will run for 0.8 sec creating task 8 task 8 will run for 8.8 sec creating task 9 task 9 will run for 0.9 sec creating task 10 task 10 will run for 4.0 sec waiting for all tasks to complete ..etc...
whilst solaris gives
201[8:25]tone@solaris-box ~ > python /usr/local/lib/python2.1/test/test_thread.py creating task 1 creating task 2 creating task 3 creating task 4 creating task 5 creating task 6 creating task 7 creating task 8 creating task 9 creating task 10 waiting for all tasks to complete ..etc...
I have done the following to try and sort this out.
#1 built a python 2.2b as that mentions problems with Solaris threading may have improved. It does to an extent, in that this happens;
202[8:26]tone@solaris-box ~ > python2.2-thread /usr/local/lib/python2.2/test/test_thread.py creating task 1 creating task 2 task 1 will run for 1.0 sec creating task 3 creating task 4 creating task 5 task 3 will run for 9.1 sec creating task 6 creating task 7 creating task 8 task 5 will run for 1.3 sec creating task 9 task 2 will run for 1.6 sec creating task 10 waiting for all tasks to complete
but Zope is not (yet) compatible with python2.2 (quite reasonable as python2.2 is a beta), so I have no way of checking it.
#2 grabbed (what I think) is the relevant parts of the 2.2 source tree and built a hybrid 2.1 python. That didn't work.
#3 inserted time.sleep(0.01) calls anywhere I could see a thread_create (can't remember the exact method) call in a clean 2.4.1 Zope installation and used the python2.1 from #2. That didn't work.
#3 installed a whole host of Solaris patches on a spare Solaris 8 box that even remotely mentioned threads, built a clean python2.1, zope 2.4.1 and tried that out (that was my Friday night high-jinks - woo-hoo). That didn't work either.
(this is starting to sound like the Monty Python 'cardboard box in the road' sketch).
So, I'm at a loss here. I may well be doing something unutterably stupid, but I'm not too sure that I'm not.
I also wouldn't believe the Sun guy, sometimes they are *so* smug. A combination of getting threading and CST via an external file storage working (I freely admit I'm at fault on the latter) would increase our throughput by a conservative factor of 24 (4 threads * 6 processors), so come on board!.
I'd be interested in seeing what a Forte compiled python was capable of. If it has threading sorted out and is just a few %-age points faster than gcc-built python, I'd *really* like to download it!.
Not a lot of help I know, but it's better you know the tiger whose tail you're tweaking, no? :)
hth, tone
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )