[Zope] Building Zope w/ Sun Forte (rather than gcc)
Paul Horbal
horbal@vlsi.enel.ucalgary.ca
Sat, 1 Dec 2001 19:32:48 -0700
Tony et al,
I went to those threads you originally mentioned about threading problems and tried some of the test scripts there. Specifically, I tried Oliver's script and I also ran Dieter's sleep script. Even while running BOTH concurrently, Zope's management interface was still responsive and I was able to browse my site with seemingly no difficulty. The same applies when I run your ZopeTime script (it takes about 20 seconds on my Netra X1, btw).
So I think threading works for me. =) This is using my gcc-compiled python (--with-threads).
Building python w/ Forte is no picnic... my last build compiled fine and then the executable puked up something about not being able to find ELF. Beats me... I'll keep plugging away though.
One thing I've noticed... I ran the 'testall.py' script first using my mt python built with Forte then with the gcc build. For both pythons, the thread test did something like this:
test_thread
test_thread
creating task 1
task 1 will run for 8.3 sec
creating task 2
task 2 will run for 5.4 sec
creating task 3
task 3 will run for 3.1 sec
creating task 4
task 4 will run for 2.7 sec
creating task 5
task 5 will run for 5.9 sec
creating task 6
creating task 7
creating task 8
creating task 9
creating task 10
waiting for all tasks to complete
task 6 will run for 7.9 sec
task 7 will run for 2.5 sec
task 8 will run for 7.6 sec
task 9 will run for 5.7 sec
task 10 will run for 9.3 sec
task 7 done
task 4 done
task 3 done
task 2 done
task 9 done
task 5 done
task 8 done
task 6 done
task 1 done
task 10 done
all tasks done
That seems like it's the kind of thing you were looking for previously... Although when run independently, the thread test doesn't do that sort of thing. Interestingly, the gcc test hung once results were printed... I sent a few breaks and it gave me a traceback that seemed to indicate it had hung waiting to acquire a new thread??
At any rate, maybe test_thread.py's behaviour isn't an indication of multi-threading not working, just the scheduling... Is it possible Solaris handles thread-creation a little differently than BSD? Perhaps the overhead of creating a new thread is smaller with Solaris, so that the original thread is able to crank out a whole bunch in one shot before those threads get their turn? You mentioned that you were comparing with Mac OS X. Aqua is a huge resource hog... maybe your python process isn't getting all the CPU it wants and the new threads pop up sooner than they otherwise would? I don't know... this is all just pure speculation...
BTW, test results using Forte-compiled Python:
116 tests OK.
6 tests failed: test___all__ test_asynchat test_nis test_socket test_socketserver test_sundry
17 tests skipped: test_al test_bsddb test_cd test_cl test_dl test_gdbm test_gl test_imgfile test_largefile test_linuxaudiodev test_minidom test_openpty test_pyexpat test_sax test_sunaudiodev test_winreg test_winsound
Results using vanilla gcc python:
119 tests OK.
2 tests failed: test_nis test_socketserver
18 tests skipped: test_al test_bsddb test_cd test_cl test_dl test_gdbm test_gl test_imgfile test_largefile test_linuxaudiodev test_minidom test_openpty test_pyexpat test_sax test_sunaudiodev test_sundry test_winreg test_winsound
I'm not sure what the failed tests are for exactly... gcc chose to skip the test_sundry test that failed with forte, but I'd like to fix them up if possible. If they aren't show-stoppers... I might put my Forte-built, 10% faster python into action...
cheers,
Paul.
On Sat, Dec 01, 2001 at 10:21:12PM +0000, Tony McDonald wrote:
>
> On Saturday, December 1, 2001, at 07:38 PM, Dieter Maurer wrote:
>
> >> Well, I have similar experices in relation to Solaris including
> >> _extremely_
> >> poor performace, threading problems/strangness (that affect even
> >> console-logging), and repeated core dumps. With Tony's report about his
> >> Solaris troubles, I feel safe to exclude DCO2 as the mayor culprit.
> >> So...
> >>
> >>
> >> Would it be reasonable to conclude that it **currently is not possible
> >> to
> >> deploy a High Availability Solaris based Zope solution**?
> > I have build serveral Zope sites under Solaris (for my old employer).
> >
> > All have been slow, much slower than expected, but have been reasonably
> > stable.
> >
>
> On our Solaris 7 system, most of our sites are stable. Under Solaris 8,
> I'm getting python SIGILL core dumps.
>
> > * Some lightly loaded sites sometimes stopped answering.
> > The problem disappeared almost completely, when they were
> > regularly
> > checked.....
> >
>
> I can confirm that has happened with us too - maddeningly, it's very
> difficult to track down.
>
> > * For the main production site, Zope was much more stable than the
> > Oracle backend database. Most outages was caused by Oracle
> > problems
> > calling for frequent Oracle restarts.
> > After an Oracle restart, Zope needed to be restarted, too.
> >
>
> We don't use Oracle, MySQL works fine for the sorts of things we need to
> do.
>
> > Threading was definitely working.
> >
>
> Dieter, if you still have access to this Solaris box, can you try out
> the script that Oliver posted in the 'threading and performance' thread
> on it and see what happens?
>
> What I've done is this (and if I'm doing something wrong here I would
> really appreciate people telling me);
>
> I run this script;
> <dtml-var standard_html_header>
> <dtml-var ZopeTime>
> <br>
> <dtml-in "_.range(80)"><dtml-in "_.range(499)">X</dtml-in><br></dtml-in>
> <br>
> <dtml-var ZopeTime>
> <dtml-var standard_html_footer>
>
> it takes about 15 seconds on a small Solaris 8 box I've been testing on.
>
> I open up a management screen on Control_Panel and open up two browser
> windows *on separate browsers* (I use IE5 and Mozilla) on this script.
> Click 'view' on one, wait a few seconds, switch to the second browser,
> select 'view' and then switch to the Control_Panel / debuginfo view to
> see what's going on.
>
> output is...
> (solaris)
> 2001/12/01 21:01:09.61767 GMT
> ... xxx ...
> 2001/12/01 21:01:25.8206 GMT
>
> and
>
> 2001/12/01 21:01:27.995 GMT
> ... xxx ...
> 2001/12/01 21:01:42.5978 GMT
>
> ie the second script *waits* until the first one has finished.
>
> This does *not* happen on MacOS-X, ie
>
> 2001/12/01 22:10:13.1872 GMT
> ... xxx ...
> 2001/12/01 22:10:44.208 GMT
>
> and
>
> 2001/12/01 22:10:21.1134 GMT
> ... xxx ...
> 2001/12/01 22:10:52.157 GMT
>
> From the timestamps, you can tell its late - and I'm very tired! :)
>
> hth
> tone
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope-dev )
--
horbal@atips.ca