[Zope] Building Zope w/ Sun Forte (rather than gcc)

[Paul Horbal]

--=====================_406000187==_
Content-Type: text/plain; charset="us-ascii"; format=flowed

Well...  it looks like that Sun guy wasn't lying about performance with 
Forte...

I rebuilt Python 2.1.1 from source and ran several passes of Pystone.

With gcc, Pystone averages about 4000 pystones/sec.  With Forte, it's 
4300!  Consistently.  That's a good 7-8% increase in performance.

As for threads... both gcc and Forte compiled python seems to do the same 
thing with threads (like you described... it creates all threads, then 
waits for completion).  I've attached the output of both tests.

GCC python was compiled with the following options:

         ./configure  --with-threads

Forte python compiled with:

         ./configure  --without-gcc -with-cxx=CC --with-threads 
--prefix=/opt/python

Both python executables were 32-bit with no Ultrasparc 
optimizations...  I'm using the latest version of Forte - v6 update 2.

For fun, I was going to try building a 64-bit UltraSparc II optimized 
python (didn't work, I'm still trying though), but I did find in the Python 
source distro README, that using Sun Workshop compilers you should add the 
-Xc -mt CFLAGS.  -mt for "multi-threading".

I then went through and built another python like this.... Pystone 
benchmarks are now 4360-4380.  That's an almost 10% boost over straight gcc 
(no change in the threads test).  Looks like gcc has some work to do on Sparc.


I'd be happy to distribute the compiled versions of Python I 
have...  What's a good way to do this for maximum portability?


cheers,
Paul.



At 08:45 AM 12/1/2001 +0000, tone.mcdonald@ncl.ac.uk wrote:

>On Friday, November 30, 2001, at 08:13 PM, Paul Horbal wrote:
>
>>Has anybody attempted to do this?
>>
>>I'm finding that even though I specify CC, CXX, put the Forte directories 
>>ahead of /usr/local/bin in my path, Python is still falling back on gcc 
>>to build Zope.  Where can I modify this behaviour?
>>
>>Our Sun rep tells us that we can kiss performance goodbye using gcc...
>>Since we have Forte anyways, I'd like to try it out.
>>
>>thanks,
>>Paul.
>
>Good luck Paul - there are some issues with python under sun hw and/or 
>solaris, which inevitably lead to issues with Zope.
>
>#1: speed: the pystone benchmark on our enterprise 3500 (400 MHz 
>Ultrasparc) is about 4500 (using gcc compiler). My TiPB using MacOS-X gets 
>6500 and a PIII/700MHz gets over 10,000. This has lead to me trying to use 
>ZEO on our system (which has multiple processors), but I ran up against a 
>problem using CoreSessionTracking needing a mountable storage that I 
>couldn't get my head around.
>
>Don't get me wrong - all our Zope systems work (although I'm getting 
>python2.1 core dumps which are *maddening*).
>
>#2: threading: Zope is a multi-threaded beast and this allows it to handle 
>more than one request at a time. This does not seem to  work under 
>Solaris. Check the archives for "threading, dtml and performance". Here's 
>a URL from NIPltd
>http://zope.nipltd.com/public/lists/zope-
>archive.nsf/47ba74c812dbc5dd8025687f0024bb5f/ebe6af5e2cd7b04780256af1002b7440?
>OpenDocument&Highlight=0,threading
>and one from Activestate
>http://aspn.activestate.com/ASPN/search?query=threading&type=Archive_zope-List
>Try out the script that Oliver Erlewein posted and see what happens on 
>your own Solaris box.
>
>What seems to indicate 'good' threading is if you get this kind of output from
>
>155[8:22]tonymcd@localhost ~ % python 
>/usr/local/lib/python2.1/test/test_thread.py
>creating task 1
>task 1 will run for 1.9 sec
>creating task 2
>task 2 will run for 4.6 sec
>creating task 3
>task 3 will run for 3.0 sec
>creating task 4
>task 4 will run for 2.6 sec
>creating task 5
>task 5 will run for 8.8 sec
>creating task 6
>task 6 will run for 9.3 sec
>creating task 7
>task 7 will run for 0.8 sec
>creating task 8
>task 8 will run for 8.8 sec
>creating task 9
>task 9 will run for 0.9 sec
>creating task 10
>task 10 will run for 4.0 sec
>waiting for all tasks to complete
>..etc...
>
>whilst solaris gives
>
>201[8:25]tone@solaris-box ~ > python 
>/usr/local/lib/python2.1/test/test_thread.py
>creating task 1
>creating task 2
>creating task 3
>creating task 4
>creating task 5
>creating task 6
>creating task 7
>creating task 8
>creating task 9
>creating task 10
>waiting for all tasks to complete
>..etc...
>
>I have done the following to try and sort this out.
>
>#1 built a python 2.2b as that mentions problems with Solaris threading 
>may have improved. It does to an extent, in that this happens;
>
>202[8:26]tone@solaris-box ~ > python2.2-thread 
>/usr/local/lib/python2.2/test/test_thread.py
>creating task 1
>creating task 2
>task 1 will run for 1.0 sec
>creating task 3
>creating task 4
>creating task 5
>task 3 will run for 9.1 sec
>creating task 6
>creating task 7
>creating task 8
>task 5 will run for 1.3 sec
>creating task 9
>task 2 will run for 1.6 sec
>creating task 10
>waiting for all tasks to complete
>
>but Zope is not (yet) compatible with python2.2 (quite reasonable as 
>python2.2 is a beta), so I have no way of checking it.
>
>#2 grabbed (what I think) is the relevant parts of the 2.2 source tree and 
>built a hybrid 2.1 python. That didn't work.
>
>#3 inserted time.sleep(0.01) calls anywhere I could see a thread_create 
>(can't remember the exact method) call in a clean 2.4.1 Zope installation 
>and used the python2.1 from #2. That didn't work.
>
>#3 installed a whole host of Solaris patches on a spare Solaris 8 box that 
>even remotely mentioned threads, built a clean python2.1, zope 2.4.1 and 
>tried that out (that was my Friday night high-jinks - woo-hoo). That 
>didn't work either.
>
>(this is starting to sound like the Monty Python 'cardboard box in the 
>road' sketch).
>
>So, I'm at a loss here. I may well be doing something unutterably stupid, 
>but I'm not too sure that I'm not.
>
>I also wouldn't believe the Sun guy, sometimes they are *so* smug. A 
>combination of getting threading and CST via an external file storage 
>working (I freely admit I'm at fault on the latter) would increase our 
>throughput by a conservative factor of 24 (4 threads * 6 processors), so 
>come on board!.
>
>I'd be interested in seeing what a Forte compiled python was capable of. 
>If it has threading sorted out and is just a few %-age points faster than 
>gcc-built python, I'd *really* like to download it!.
>
>Not a lot of help I know, but it's better you know the tiger whose tail 
>you're tweaking, no? :)
>
>hth,
>tone
>
>
>_______________________________________________
>Zope maillist  -  Zope@zope.org
>http://lists.zope.org/mailman/listinfo/zope
>**   No cross posts or HTML encoding!  **
>(Related lists - http://lists.zope.org/mailman/listinfo/zope-announce
>http://lists.zope.org/mailman/listinfo/zope-dev )

--=====================_406000187==_
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: attachment; filename="fortethr.txt"

creating task 1
creating task 2
creating task 3
creating task 4
creating task 5
creating task 6
creating task 7
creating task 8
creating task 9
creating task 10
waiting for all tasks to complete
task 1 will run for 7.7 sec
task 2 will run for 8.5 sec
task 3 will run for 4.9 sec
task 4 will run for 1.8 sec
task 5 will run for 2.4 sec
task 6 will run for 5.5 sec
task 7 will run for 1.3 sec
task 8 will run for 1.1 sec
task 9 will run for 3.1 sec
task 10 will run for 0.5 sec
task 10 done
task 8 done
task 7 done
task 4 done
task 5 done
task 9 done
task 3 done
task 6 done
task 1 done
task 2 done
all tasks done

*** Barrier Test ***
task 0 will run for 0.0 sec
task 1 will run for 3.4 sec
task 2 will run for 3.2 sec
task 3 will run for 2.6 sec
task 4 will run for 3.2 sec
task 5 will run for 8.7 sec
task 6 will run for 6.1 sec
task 7 will run for 0.7 sec
task 8 will run for 4.4 sec
task 9 will run for 0.9 sec
task 0 entering barrier 0
task 7 entering barrier 0
task 9 entering barrier 0
task 3 entering barrier 0
task 2 entering barrier 0
task 4 entering barrier 0
task 1 entering barrier 0
task 8 entering barrier 0
task 6 entering barrier 0
task 5 entering barrier 0
task 0 leaving barrier 0
task 0 will run for 0.0 sec
task 5 leaving barrier 0
task 5 will run for 4.9 sec
task 7 leaving barrier 0
task 7 will run for 0.1 sec
task 9 leaving barrier 0
task 9 will run for 0.0 sec
task 3 leaving barrier 0
task 3 will run for 8.3 sec
task 2 leaving barrier 0
task 2 will run for 4.8 sec
task 4 leaving barrier 0
task 4 will run for 6.4 sec
task 1 leaving barrier 0
task 1 will run for 1.0 sec
task 8 leaving barrier 0
task 8 will run for 5.8 sec
task 6 leaving barrier 0
task 6 will run for 7.0 sec
task 0 entering barrier 1
task 9 entering barrier 1
task 7 entering barrier 1
task 1 entering barrier 1
task 2 entering barrier 1
task 5 entering barrier 1
task 8 entering barrier 1
task 4 entering barrier 1
task 6 entering barrier 1
task 3 entering barrier 1
task 0 leaving barrier 1
task 0 will run for 0.0 sec
task 3 leaving barrier 1
task 3 will run for 4.9 sec
task 7 leaving barrier 1
task 7 will run for 4.6 sec
task 9 leaving barrier 1
task 9 will run for 0.7 sec
task 1 leaving barrier 1
task 1 will run for 6.3 sec
task 2 leaving barrier 1
task 2 will run for 5.4 sec
task 5 leaving barrier 1
task 5 will run for 8.5 sec
task 8 leaving barrier 1
task 8 will run for 2.1 sec
task 4 leaving barrier 1
task 4 will run for 1.3 sec
task 6 leaving barrier 1
task 6 will run for 9.3 sec
task 0 entering barrier 2
task 9 entering barrier 2
task 4 entering barrier 2
task 8 entering barrier 2
task 7 entering barrier 2
task 3 entering barrier 2
task 2 entering barrier 2
task 1 entering barrier 2
task 5 entering barrier 2
task 6 entering barrier 2
task 0 leaving barrier 2
task 9 leaving barrier 2
task 6 leaving barrier 2
task 4 leaving barrier 2
task 7 leaving barrier 2
task 8 leaving barrier 2
task 3 leaving barrier 2
task 2 leaving barrier 2
task 1 leaving barrier 2
task 5 leaving barrier 2
all tasks done

--=====================_406000187==_
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: attachment; filename="gccthr.txt"

creating task 1
creating task 2
creating task 3
creating task 4
creating task 5
creating task 6
creating task 7
creating task 8
creating task 9
creating task 10
waiting for all tasks to complete
task 1 will run for 6.2 sec
task 2 will run for 9.8 sec
task 3 will run for 3.1 sec
task 4 will run for 4.4 sec
task 5 will run for 1.5 sec
task 6 will run for 5.1 sec
task 7 will run for 3.7 sec
task 8 will run for 7.8 sec
task 9 will run for 2.0 sec
task 10 will run for 9.5 sec
task 5 done
task 9 done
task 3 done
task 7 done
task 4 done
task 6 done
task 1 done
task 8 done
task 10 done
task 2 done
all tasks done

*** Barrier Test ***
task 0 will run for 0.0 sec
task 1 will run for 7.9 sec
task 2 will run for 4.5 sec
task 3 will run for 7.5 sec
task 4 will run for 8.9 sec
task 5 will run for 5.8 sec
task 6 will run for 7.7 sec
task 7 will run for 7.6 sec
task 8 will run for 8.2 sec
task 9 will run for 5.7 sec
task 0 entering barrier 0
task 2 entering barrier 0
task 9 entering barrier 0
task 5 entering barrier 0
task 3 entering barrier 0
task 7 entering barrier 0
task 6 entering barrier 0
task 1 entering barrier 0
task 8 entering barrier 0
task 4 entering barrier 0
task 4 leaving barrier 0
task 4 will run for 1.4 sec
task 0 leaving barrier 0
task 0 will run for 0.0 sec
task 2 leaving barrier 0
task 2 will run for 1.8 sec
task 9 leaving barrier 0
task 9 will run for 3.9 sec
task 5 leaving barrier 0
task 5 will run for 1.9 sec
task 3 leaving barrier 0
task 3 will run for 8.2 sec
task 7 leaving barrier 0
task 7 will run for 9.3 sec
task 6 leaving barrier 0
task 6 will run for 9.9 sec
task 1 leaving barrier 0
task 1 will run for 2.5 sec
task 8 leaving barrier 0
task 8 will run for 6.6 sec
task 0 entering barrier 1
task 4 entering barrier 1
task 2 entering barrier 1
task 5 entering barrier 1
task 1 entering barrier 1
task 9 entering barrier 1
task 8 entering barrier 1
task 3 entering barrier 1
task 7 entering barrier 1
task 6 entering barrier 1
task 6 leaving barrier 1
task 6 will run for 6.6 sec
task 0 leaving barrier 1
task 0 will run for 0.0 sec
task 4 leaving barrier 1
task 4 will run for 1.2 sec
task 2 leaving barrier 1
task 2 will run for 3.0 sec
task 5 leaving barrier 1
task 5 will run for 4.0 sec
task 1 leaving barrier 1
task 1 will run for 6.5 sec
task 9 leaving barrier 1
task 9 will run for 0.3 sec
task 8 leaving barrier 1
task 8 will run for 0.8 sec
task 3 leaving barrier 1
task 3 will run for 2.1 sec
task 7 leaving barrier 1
task 7 will run for 0.4 sec
task 0 entering barrier 2
task 9 entering barrier 2
task 7 entering barrier 2
task 8 entering barrier 2
task 4 entering barrier 2
task 3 entering barrier 2
task 2 entering barrier 2
task 5 entering barrier 2
task 1 entering barrier 2
task 6 entering barrier 2
task 6 leaving barrier 2
task 0 leaving barrier 2
task 9 leaving barrier 2
task 7 leaving barrier 2
task 8 leaving barrier 2
task 4 leaving barrier 2
task 3 leaving barrier 2
task 2 leaving barrier 2
task 5 leaving barrier 2
task 1 leaving barrier 2
all tasks done

--=====================_406000187==_
Content-Type: text/plain; charset="us-ascii"; format=flowed


--=====================_406000187==_--