Thanks Tim, I definitely understand your point and knowing about these issues is certainly useful. Maybe mentioning the unit tests muddied my original question. My practical reality is a production environment in which a number of applications are already in heavy use, running happily against python2.3.2. I want to run an up-to-date Zope2 that can talk to these applications (i.e. as a frontend utility), without the considerable block of setting up a whole new python build and upsetting the otherwise-placid state of affairs. What's confusing, though not exactly critical, is why source distributions seem to allow for some leeway as far as which python version you build Zope onto, but binaries have this less forgiving posture. The less-forgiving approach implies that Zope will not work with an earlier py version, but that doesn't exactly seem to be the case (it works, just 'sub-optimally'). Jim -----Original Message----- From: Tim Peters [mailto:tim.peters@gmail.com] Sent: Tuesday, August 24, 2004 5:05 PM To: Jim Abramson Cc: zope@zope.org Subject: Re: [Zope] Zope2.7.2 with ActivePython2.3.2 [Jim Abramson]
As a small aside to this topic, I just ran the (entire?) suite of unit
tests - 2586 of them - with Zope2.7.2 using an ActivePython2.3.2 binary install (pretty much out of the box) on WinXP. I realize this isn't everything as far as a full compatibility assessment goes - but nothing erred.
Bugfix releases of Python strive not to change visible behavior, so that's not surprising. Later versions of Python are generally required because of bugs in Python that afffected Zope, often in endcases so bizarre (Zope plays in Python's dark corners like few other apps dare) that only one way to trigger it is ever discovered. Tests to ensure that such bugs are fixed end up in Python's test suite, not in Zope's -- because they're Python bugs, not Zope bugs. For example, here are a subset of items fixed in Python 2.3.3 that were discovered in Zope (2 or 3): - Critical bugfix, for SF bug 839548: if a weakref with a callback, its callback, and its weakly referenced object, all became part of cyclic garbage during a single run of garbage collection, the order in which they were torn down was unpredictable. It was possible for the callback to see partially-torn-down objects, leading to immediate segfaults, or, if the callback resurrected garbage objects, to resurrect insane objects that caused segfaults (or other surprises) later. In one sense this wasn't surprising, because Python's cyclic gc had no knowledge of Python's weakref objects. It does now. When weakrefs with callbacks become part of cyclic garbage now, those weakrefs are cleared first. The callbacks don't trigger then, preventing the problems. If you need callbacks to trigger, then just as when cyclic gc is not involved, you need to write your code so that weakref objects outlive the objects they weakly reference. - Critical bugfix, for SF bug 840829: if cyclic garbage collection happened to occur during a weakref callback for a new-style class instance, subtle memory corruption could result (in a release build; in a debug build, a segfault occurred reliably very soon after). This has been repaired. - At Python shutdown time (Py_Finalize()), 2.3 called cyclic garbage collection twice, both before and after tearing down modules. The call after tearing down modules has been disabled, because too much of Python has been torn down then for __del__ methods and weakref callbacks to execute sanely. The most common symptom was a sequence of uninformative messages on stderr when Python shut down, produced by threads trying to raise exceptions, but unable to report the nature of their problems because too much of the sys module had already been destroyed. There are more in 2.3.4: - Fixed problem where PyWeakref_NewRef() and PyWeakref_NewProxy() assumed that initial existing entries in an object's weakref list would not be removed while allocating a new weakref object. Since GC could be invoked at that time, however, that assumption was invalid. In a truly obscure case of GC being triggered during creation for a new weakref object for an referent which already has a weakref without a callback which is only referenced from cyclic trash, a memory error can occur. This consistently created a segfault in a debug build, but provided less predictable behavior in a release build. - Fixed a bug in object.__reduce_ex__ when using protocol 2. Failure to clear the error when attempts to get the __getstate__ attribute fail caused intermittent errors and odd behavior. You're insane if you think you want to track down one of those nightmares yourself all over again <wink>.
[Jim Abramson]
I definitely understand your point and knowing about these issues is certainly useful. Maybe mentioning the unit tests muddied my original question.
My practical reality is a production environment in which a number of applications are already in heavy use, running happily against python2.3.2. I want to run an up-to-date Zope2 that can talk to these applications (i.e. as a frontend utility), without the considerable block of setting up a whole new python build and upsetting the otherwise-placid state of affairs.
You can do that, but then you're on your own for support. Zope Corp sometimes pays me to track down and fix Python bugs that affect Zope, but they're not going to pay me to track them down a second time.
What's confusing, though not exactly critical, is why source distributions seem to allow for some leeway as far as which python version you build Zope onto, but binaries have this less forgiving posture. The less-forgiving approach implies that Zope will not work with an earlier py version, but that doesn't exactly seem to be the case (it works, just 'sub-optimally').
After you experience a segfault under heavy load, I doubt you'll be posting a message here titled Help with sub-optimal memory behavior? <wink>. Most Zope users on WIndows don't even have C compilers, so Zope on WIndows ships with precompiled binaries for everything Zope needs, including Python. So it goes. You can build Zope from source on Windows too if you want to, although it's an undertaking. Note that the Zope 2.7.2 info page: http://zope.org/Products/Zope/2.7.2/Zope-2_7_2-released lists Python 2.3.3 as the minimum supported version. That at least means nobody at Zope Corp will pay attention if you try to report a bug against Zope using an earlier version. Zope 2.8 will require 2.3.4 minimum (because of the Zope-critical bugs still in 2.3.3 I listed last time, which affect code Zope 2.8 inherits from the Zope 3 project).
I think this illustrates the same scenario that some of us use to face from time to time. Everytime a Zope 2.X.0 version is out we force ourselves to install it and have it running locally for trying its new features, to test its compatibility with old apps, and eventually to run a copy of our production stuff - in search of possible bugs -, but definitely not to air it. A brand new Zope 2.X.X is different. We supose that chances to bugs are fewer than in a 2.X.0 release and so some of us tend to believe its time to try it on air, although we knew we'll expect random bugs. Hence I see no problem in matching a specific release of Zope with a Python one, but, according to my point, the fact of to find a bug in a 2.X.X version that wasn't in its corresponding 2.X.0 one (not even in any previous release) may become misleading in the long term, IMHO. I do have a bug report that I will post as soon as I get to the office (that's not the important stuff in this comment) but in any case, as I'm definitely not complaining of the formidable job of Zope developers, my call is to keep Zope's releasing on a slower pace. Maybe we simply should avoid to call the latest release the stable one. Ausum ----- Original Message ----- From: "Tim Peters" <tim.peters@gmail.com> To: "Jim Abramson" <jabramson@wgen.net> Cc: <zope@zope.org> Sent: Tuesday, August 24, 2004 6:05 PM Subject: Re: [Zope] Zope2.7.2 with ActivePython2.3.2
[Jim Abramson]
I definitely understand your point and knowing about these issues is certainly useful. Maybe mentioning the unit tests muddied my original question.
My practical reality is a production environment in which a number of applications are already in heavy use, running happily against python2.3.2. I want to run an up-to-date Zope2 that can talk to these applications (i.e. as a frontend utility), without the considerable block of setting up a whole new python build and upsetting the otherwise-placid state of affairs.
You can do that, but then you're on your own for support. Zope Corp sometimes pays me to track down and fix Python bugs that affect Zope, but they're not going to pay me to track them down a second time.
(...)
participants (3)
-
Ausum Studio -
Jim Abramson -
Tim Peters