Zope2: Reducing C dependencies
Hi. For Zope 2.13 (trunk) I'd like to try to reduce the C dependencies of the Zope2 distribution itself. Ideally it would not have any C dependencies in its own distribution, but that's a bit too much for now. As a next step I'd like to split out the initgroups, Missing, MultiMapping, Record and ThreadLock packages into their own distributions. The code in these hasn't changed in years and I don't see this changing anytime soon. If this is done the remaining C libraries are in AccessControl, DocumentTemplate and ZCTextIndex. AccessControl has a reference implementation in Python and could the C library could be made an optional optimization. The ZCTextIndex okascore.c and stopper.c should also be relatively easy to do as Python implementations. DocumentTemplate has a pDocumentTemplate.py module as a reference implementation, which clearly states that it is outdated. This would need to be updated to match the C implementation. Hanno
Hanno Schlichting wrote:
Hi.
For Zope 2.13 (trunk) I'd like to try to reduce the C dependencies of the Zope2 distribution itself. Ideally it would not have any C dependencies in its own distribution
Why? Martin (who spent all day trying to get Zope 2 to build on SuSE Linux before realising it had a b0rked libpython2.6 installed) -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Tue, Mar 30, 2010 at 5:26 PM, Martin Aspeli <optilude+lists@gmail.com> wrote:
Hanno Schlichting wrote:
For Zope 2.13 (trunk) I'd like to try to reduce the C dependencies of the Zope2 distribution itself. Ideally it would not have any C dependencies in its own distribution
Why?
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier. Hanno
Hanno Schlichting wrote:
On Tue, Mar 30, 2010 at 5:26 PM, Martin Aspeli<optilude+lists@gmail.com> wrote:
Hanno Schlichting wrote:
For Zope 2.13 (trunk) I'd like to try to reduce the C dependencies of the Zope2 distribution itself. Ideally it would not have any C dependencies in its own distribution Why?
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier.
+1 I'm generally for it, we just need to (a) make sure we're not going to negatively impact performance if we can help it and (b) explain our rationale. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
* 2010-03-30 17:44, Martin Aspeli wrote:
I'm generally for it, we just need to (a) make sure we're not going to negatively impact performance if we can help it and (b) explain our rationale.
Not sure I understood correctly what Hanno proposed, but we are not going to negatively impact performance because we are not replacing C code with Python code, just factoring it out into an independent package. Right? Best regards. Fabio
On Tue, Mar 30, 2010 at 5:44 PM, Martin Aspeli <optilude+lists@gmail.com> wrote:
Hanno Schlichting wrote:
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier.
+1
I'm generally for it, we just need to (a) make sure we're not going to negatively impact performance if we can help it and (b) explain our rationale.
This is all going to be transparent to everyone. We've already split out Acquisition, ExtensionClass (ComputedAttribute, MethodObject) in the same way. Note that the packages I'm proposing to split out have no Python fallback. The C code which we might want to turn into optional optimizations at a later point, would follow the same standard we already use in the Zope Toolkit. So there won't be any new kind of risk. For example zope.i18nmessageid and zope.interface use this approach. Hanno
On 3/30/10 18:05 , Hanno Schlichting wrote:
The C code which we might want to turn into optional optimizations at a later point, would follow the same standard we already use in the Zope Toolkit. So there won't be any new kind of risk. For example zope.i18nmessageid and zope.interface use this approach.
If I remember corrcetly Malthe's pure-python variant of zope.i18nmessage got vetoed late 2008, so it is still requires C compilation. Wichert.
On Tue, Mar 30, 2010 at 7:54 PM, Wichert Akkerman <wichert@wiggy.net> wrote:
If I remember corrcetly Malthe's pure-python variant of zope.i18nmessage got vetoed late 2008, so it is still requires C compilation.
Look at the source [1] or the release notes for the 3.5.0 release from last June. That old story has long been resolved. No need to bring it up again. Hanno [1] http://svn.zope.org/zope.i18nmessageid/trunk/setup.py?rev=103768&view=markup
Am 30.03.2010, 17:31 Uhr, schrieb Hanno Schlichting <hanno@hannosch.eu>:
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier.
+1 in general but -1 if it is really just about windows binaries. I like the idea of cleaning up dependencies and optional C-optimisations but as a general process. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
Charlie Clark wrote:
Am 30.03.2010, 17:31 Uhr, schrieb Hanno Schlichting<hanno@hannosch.eu>:
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier.
+1 in general but -1 if it is really just about windows binaries.
Why -1 if it's just about windows binaries? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
Am 31.03.2010, 13:54 Uhr, schrieb Martin Aspeli <optilude+lists@gmail.com>:
Why -1 if it's just about windows binaries?
Because I don't think that, if this were the case, this would be the best solution to the problem. But I believe that there will probably be a benefit for all systems, especially buildbots if the C extensions don't need compiling every time. My comment was similar to your initial question an attempt to clarify the why. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
Charlie Clark wrote:
Am 31.03.2010, 13:54 Uhr, schrieb Martin Aspeli<optilude+lists@gmail.com>:
Why -1 if it's just about windows binaries?
Because I don't think that, if this were the case, this would be the best solution to the problem.
What would be a better solution? Windows is sometimes treated as a second class citizen development platform, probably due to developer preference (my own included). There are a lot of people who either prefer Windows or have no choice whether to use it, though. If we can serve them better, with more timely releases that deviate less from our other code, that's a huge win, and reason enough to do this in itself, IMHO. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Wednesday 31 March 2010, Martin Aspeli wrote:
Windows is sometimes treated as a second class citizen development platform, probably due to developer preference (my own included). There are a lot of people who either prefer Windows or have no choice whether to use it, though. If we can serve them better, with more timely releases that deviate less from our other code, that's a huge win, and reason enough to do this in itself, IMHO.
I agree. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Wednesday 31 March 2010, Martin Aspeli wrote:
It simplifies the release process for Zope2. In most cases upgrading to a new version of Zope2 won't involve any changes to C code. If the C code is split out, we won't have to release any new Windows binary eggs of any kind. Upgrades on systems without a compiler are also going to be easier.
+1 in general but -1 if it is really just about windows binaries.
Why -1 if it's just about windows binaries?
A big +1 for me. If splitting these out means we do not have to produce binary Zope 2 releases, then that's fantastic! Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On 3/30/10 16:56 , Hanno Schlichting wrote:
Hi.
For Zope 2.13 (trunk) I'd like to try to reduce the C dependencies of the Zope2 distribution itself. Ideally it would not have any C dependencies in its own distribution, but that's a bit too much for now.
As a next step I'd like to split out the initgroups, Missing, MultiMapping, Record and ThreadLock packages into their own distributions. The code in these hasn't changed in years and I don't see this changing anytime soon.
If this is done the remaining C libraries are in AccessControl, DocumentTemplate and ZCTextIndex. AccessControl has a reference implementation in Python and could the C library could be made an optional optimization. The ZCTextIndex okascore.c and stopper.c should also be relatively easy to do as Python implementations.
All of those are quite stable as well, aren't they? What is the rationale for not splitting those out? Wichert.
On Tue, Mar 30, 2010 at 5:49 PM, Wichert Akkerman <wichert@wiggy.net> wrote:
On 3/30/10 16:56 , Hanno Schlichting wrote:
If this is done the remaining C libraries are in AccessControl, DocumentTemplate and ZCTextIndex. AccessControl has a reference implementation in Python and could the C library could be made an optional optimization. The ZCTextIndex okascore.c and stopper.c should also be relatively easy to do as Python implementations.
All of those are quite stable as well, aren't they? What is the rationale for not splitting those out?
These packages have hairy interdependencies on lots of other things. Like code in AccessControl modifies code inside DocumentTemplate, DocumentTemplate imports from AccessControl at the same time. Or Zope2.App needs HelpSys which in turn relies on ZCTextIndex. There's many more of these examples. Before we could think about splitting out any of these, we'd need to clean up those dependencies. At least AccessControl has seen a lot of changes in current Zope trunk compared to 2.12.x, so I'd be hesitant to split it out right now. Hanno
participants (6)
-
Charlie Clark -
Fabio Tranchitella -
Hanno Schlichting -
Martin Aspeli -
Stephan Richter -
Wichert Akkerman