Proposal: Scrap zpkg for Zope2 releases
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I worked last night with folks from the Fedora Extras project who were trying to package Zope 2.9.3 for FC5+. Because they were working from the release tarball, generated by 'zpkg', much of my knowledge about how the build process works (or doesn't) was invalid: - The Makefile generated by 'zpkg' does not have bugfixes / features which have been made to the 'Makefile' created by 'configure' in a checkout. - The 'install.py' script has subtly-different semantics from the 'setup.py' script in the checkout. In particular, it was hard to figure out how to get the installed libraries correct for the x86_64 package. - We have had a bunch of bugs since 2.9 related to the 'zpkg'-based build, some related to lost features and other to various kinds of breakage (see #1967, #1968, #1996, #2030, #2081, #2082, #2083, #2121). - Working inside the 'zpkg'-generated tarball is *very* confusing, even for experienced Zope developers: "Where is the source?" is a frequent cry in such cases. All of this is due to the fact that none of the maintainers of Zope2 is also a conusmer of the zpkg-gnereated releases; those consumers are the downstream packagers and sysadmins who have no idea how to work in that setup, and who can't even (easily) get help on it from the Zope developer community. I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases. Instead, we should recreate the version of the 'inst' stuff removed in the 2.9 beta cycle, and update it for any changes to the tree made since then. I volunteer to do the work, assuming the community concurs. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEmqrH+gerLs4ltQ4RApF2AKDTWq8XqY4OCuj5BpPZ3omCpnzEtwCghPnO nZ8S8NqTC1oZx8o3KVVJxBo= =4qXe -----END PGP SIGNATURE-----
On 6/22/06, Tres Seaver <tseaver@palladion.com> wrote:
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases. Instead, we should recreate the version of the 'inst' stuff removed in the 2.9 beta cycle, and update it for any changes to the tree made since then.
+1 -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Every sin is the result of a collaboration." --Lucius Annaeus Seneca
Tres Seaver wrote:
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases.
+1
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 22 Jun 2006, at 16:35, Tres Seaver wrote:
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases. Instead, we should recreate the version of the 'inst' stuff removed in the 2.9 beta cycle, and update it for any changes to the tree made since then.
+1 Thanks Tres, I think many users will be happy about this. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFEmtKzRAx5nvEhZLIRAojxAJ91LchtGqtffvleND/SMuN6m0c05gCfUnwd bwJetqs36Vwq4rnhNbEz1wU= =TcJf -----END PGP SIGNATURE-----
Tres Seaver wrote at 2006-6-22 10:35 -0400:
... I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases.
I have never worked with "zpkg" but have read numerous messages about problems. I, for my part, am therefore in favour of your proposal. -- Dieter
+1, back to the future... I'll note that when I was getting this stuff done in 2.7, it was incredibly useful to try to package Zope as an RPM (or deb, etc) while doing itbecause you find out where all the pointy edges are. Some of the functionality of the install routine (copyzopeskel in particular) was driven by a desire to make Zope easier to downstream- package. - C On Jun 22, 2006, at 10:35 AM, Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I worked last night with folks from the Fedora Extras project who were trying to package Zope 2.9.3 for FC5+. Because they were working from the release tarball, generated by 'zpkg', much of my knowledge about how the build process works (or doesn't) was invalid:
- The Makefile generated by 'zpkg' does not have bugfixes / features which have been made to the 'Makefile' created by 'configure' in a checkout.
- The 'install.py' script has subtly-different semantics from the 'setup.py' script in the checkout. In particular, it was hard to figure out how to get the installed libraries correct for the x86_64 package.
- We have had a bunch of bugs since 2.9 related to the 'zpkg'-based build, some related to lost features and other to various kinds of breakage (see #1967, #1968, #1996, #2030, #2081, #2082, #2083, #2121).
- Working inside the 'zpkg'-generated tarball is *very* confusing, even for experienced Zope developers: "Where is the source?" is a frequent cry in such cases.
All of this is due to the fact that none of the maintainers of Zope2 is also a conusmer of the zpkg-gnereated releases; those consumers are the downstream packagers and sysadmins who have no idea how to work in that setup, and who can't even (easily) get help on it from the Zope developer community.
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases. Instead, we should recreate the version of the 'inst' stuff removed in the 2.9 beta cycle, and update it for any changes to the tree made since then.
I volunteer to do the work, assuming the community concurs.
Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFEmqrH+gerLs4ltQ4RApF2AKDTWq8XqY4OCuj5BpPZ3omCpnzEtwCghPnO nZ8S8NqTC1oZx8o3KVVJxBo= =4qXe -----END PGP SIGNATURE-----
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Tres Seaver wrote: [scrap zpkg for Zope 2] +1 Regards, Martijn
On 6/22/06, Tres Seaver <tseaver@palladion.com> wrote:
I volunteer to do the work, assuming the community concurs.
In that case. +several million. :) How you would move Zope2 to eggs is beyond me, but zpkg is confusing and too tersely documented. At least we should move to some sort of standard package, and then extend them if needed. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
I worked last night with folks from the Fedora Extras project who were trying to package Zope 2.9.3 for FC5+. Because they were working from the release tarball, generated by 'zpkg', much of my knowledge about how the build process works (or doesn't) was invalid:
- The Makefile generated by 'zpkg' does not have bugfixes / features which have been made to the 'Makefile' created by 'configure' in a checkout.
- The 'install.py' script has subtly-different semantics from the 'setup.py' script in the checkout. In particular, it was hard to figure out how to get the installed libraries correct for the x86_64 package.
- We have had a bunch of bugs since 2.9 related to the 'zpkg'-based build, some related to lost features and other to various kinds of breakage (see #1967, #1968, #1996, #2030, #2081, #2082, #2083, #2121).
- Working inside the 'zpkg'-generated tarball is *very* confusing, even for experienced Zope developers: "Where is the source?" is a frequent cry in such cases.
All of this is due to the fact that none of the maintainers of Zope2 is also a conusmer of the zpkg-gnereated releases; those consumers are the downstream packagers and sysadmins who have no idea how to work in that setup, and who can't even (easily) get help on it from the Zope developer community.
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs, and that we should thus retire zpkg as the means for building Zope2 releases. Instead, we should recreate the version of the 'inst' stuff removed in the 2.9 beta cycle, and update it for any changes to the tree made since then.
I volunteer to do the work, assuming the community concurs.
I will be ready shortly to merge this branch to the 2.9 branch, the 2.10 branch, and the Zope2 trunk. Here is how I have tested it so far: - All unit tests pass, with the same count (and deprecation warnings!) in my sandbox for this change as for the 2.9 head. - I have made an 'sdist' tarball from the sandboxy, and then been able to run './configure ... && make && makd test && make install' from the directory in which I unpacked the tarball. - I was also able to create an out-of-tree build from that unpacked directory. - Instances created from the tarball can run unit tests for their products via 'bin/zopectl test'. - The site starts up fine, and I can add content via the ZMI, etc. Remaining TODOs: - I would like to have a few more folks try out working with the tarball, which I have uploaded to zope.org: http://www.zope.org/Members/tseaver/Zope-2.9.4-retire_zpkg.tgz - I should find and update any documentation on making a release to reflect the new process. I don't think any of that documentation is in the source tree: does anyone have a pointer to it? - Make the corresponding changes to the Windows build machinery. Comments? Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEnUDU+gerLs4ltQ4RAr+hAJ9bw9T1PaBTc6JDUNdCdQTYVuFKcQCfcDFB Qe7cCO/QmBnF/92vPaPuFO8= =OEKK -----END PGP SIGNATURE-----
Thanks a lot for your work! --On 24. Juni 2006 09:40:36 -0400 Tres Seaver <tseaver@palladion.com> wrote:
Remaining TODOs:
- I would like to have a few more folks try out working with the tarball, which I have uploaded to zope.org:
http://www.zope.org/Members/tseaver/Zope-2.9.4-retire_zpkg.tgz
Works fine for me. I don't see any issues
- Make the corresponding changes to the Windows build machinery.
Perhaps we should check the Windows build machinery before merging the branch. It's easy to mess up the Windows stuff but it's hard to fix it :-) Andreas
On Sat, Jun 24, 2006 at 04:08:23PM +0200, Andreas Jung wrote: | > - Make the corresponding changes to the Windows build machinery. | > | | Perhaps we should check the Windows build machinery before merging the | branch. | It's easy to mess up the Windows stuff but it's hard to fix it :-) I can help some here. After all, building Zope on Windows fills 60% of my days lately so I'm very interested that this is as easy and has as little dependencies as possible. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sidnei da Silva wrote:
On Sat, Jun 24, 2006 at 04:08:23PM +0200, Andreas Jung wrote: | > - Make the corresponding changes to the Windows build machinery. | > | | Perhaps we should check the Windows build machinery before merging the | branch. | It's easy to mess up the Windows stuff but it's hard to fix it :-)
I can help some here. After all, building Zope on Windows fills 60% of my days lately so I'm very interested that this is as easy and has as little dependencies as possible.
I don't grok what the current Windows build process is. Could you point me at a writeup? Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEncUm+gerLs4ltQ4RArZAAJoDiGTlsLRhTzYPARBLvCOtSrhNVQCeLm+0 J9gazK4WWSPIfsdrKQ++/QA= =E9pk -----END PGP SIGNATURE-----
On Sat, Jun 24, 2006 at 07:05:10PM -0400, Tres Seaver wrote: | -----BEGIN PGP SIGNED MESSAGE----- | Hash: SHA1 | | Sidnei da Silva wrote: | > On Sat, Jun 24, 2006 at 04:08:23PM +0200, Andreas Jung wrote: | > | > - Make the corresponding changes to the Windows build machinery. | > | > | > | | > | Perhaps we should check the Windows build machinery before merging the | > | branch. | > | It's easy to mess up the Windows stuff but it's hard to fix it :-) | > | > I can help some here. After all, building Zope on Windows fills 60% of | > my days lately so I'm very interested that this is as easy and has as | > little dependencies as possible. | | I don't grok what the current Windows build process is. Could you point | me at a writeup? Somewhere below inst/WinBuilders there's a README.txt. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
Tres Seaver wrote:
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs,
Where are the eggs, btw?
I will be ready shortly to merge this branch to the 2.9 branch, the 2.10 branch, and the Zope2 trunk. Here is how I have tested it so far:
- All unit tests pass, with the same count (and deprecation warnings!) in my sandbox for this change as for the 2.9 head.
This doesn't surprise me. After all, you just tarred up the SVN export. zpkg actually only tars up what is explicitly selected to be released. Your tarball contains a few things that haven't been in Zope 2.9.x releases before, e.g.: - zope.app.boston - zope.app.catalog - zope.app.css - zope.app.dav - zope.app.debugskin - zope.app.demo - zope.app.fssync - zope.app.homefolder - zope.app.i18nfile - zope.app.interpreter - zope.app.locking - zope.app.module - zope.app.observable - zope.app.pluggableauth - zope.app.presentation - zope.app.pythonpage - zope.app.recorder - zope.app.schemacontent - zope.app.securitypolicy - zope.app.server - zope.app.sqlexpr - zope.app.styleguide - zope.app.twisted - zope.app.versioncontrol - zope.app.wfmc - zope.app.winservice - zope.app.workflow - zope.app.zopetop Most of these a) have either never released even in Zope 3 (this is the majority), b) are sample things (like css, styleguide, demo, etc.) that are (for the most part) bitrotting in zope.app c) are so much core to Zope 3 that they don't make sense in Zope 2 (such as twisted, securitypolicy) While b) and c) might not do much harm, the fact that things in a) *are* released now *does* make a new feature. Even worse, most of the things in category a) *should not* be release because they're considered broken or unstable. I agree that zope.app is horribly trashed with junk that doesn't belong in there (probably doesn't even belong inside the Zope 3 tree), but that's how it is right now. Just dumping it in a tarball isn't the way to go... If it had been that trivial to NOT use zpkg back then, I certainly wouldn't have considered it at all... Philipp
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Philipp von Weitershausen wrote:
Tres Seaver wrote:
I believe that the extra flexibility which zpkg is intended to provide (dependency-based subset distributions, primarily) would be better served by moving Zope to use eggs,
Where are the eggs, btw?
Not part of this proposal. I just want to use the same installation machinery as Zope2 has always used, and let the egg stuff sort out later.
I will be ready shortly to merge this branch to the 2.9 branch, the 2.10 branch, and the Zope2 trunk. Here is how I have tested it so far:
- All unit tests pass, with the same count (and deprecation warnings!) in my sandbox for this change as for the 2.9 head.
This doesn't surprise me. After all, you just tarred up the SVN export. zpkg actually only tars up what is explicitly selected to be released. Your tarball contains a few things that haven't been in Zope 2.9.x releases before, e.g.:
- zope.app.boston - zope.app.catalog - zope.app.css - zope.app.dav - zope.app.debugskin - zope.app.demo - zope.app.fssync - zope.app.homefolder - zope.app.i18nfile - zope.app.interpreter - zope.app.locking - zope.app.module - zope.app.observable - zope.app.pluggableauth - zope.app.presentation - zope.app.pythonpage - zope.app.recorder - zope.app.schemacontent - zope.app.securitypolicy - zope.app.server - zope.app.sqlexpr - zope.app.styleguide - zope.app.twisted - zope.app.versioncontrol - zope.app.wfmc - zope.app.winservice - zope.app.workflow - zope.app.zopetop
Most of these
a) have either never released even in Zope 3 (this is the majority),
b) are sample things (like css, styleguide, demo, etc.) that are (for the most part) bitrotting in zope.app
c) are so much core to Zope 3 that they don't make sense in Zope 2 (such as twisted, securitypolicy)
While b) and c) might not do much harm, the fact that things in a) *are* released now *does* make a new feature. Even worse, most of the things in category a) *should not* be release because they're considered broken or unstable. I agree that zope.app is horribly trashed with junk that doesn't belong in there (probably doesn't even belong inside the Zope 3 tree), but that's how it is right now. Just dumping it in a tarball isn't the way to go...
If it had been that trivial to NOT use zpkg back then, I certainly wouldn't have considered it at all...
The point is that the "release" tarball should generate the same environment that the developers routinely work in; otherwise, we leave the poor suckers who install from it stuck with whatever bugs are caused by the difference. I'll agree that these items ought not to be in the Zope2 tree: fixing that *in the checkout* would be a worthwhile effort. The simplest approach I can see to that would be: - On the Zope3 side, make it a rule that 'zope.app' contains only packages, not modules, converting the existing modules into packages with '__init__.py' corresponding to the current modules. Move the corresponding tests down into the new packages, as well. Currently, that would involve changing the following: o zope.app.datetimeutils o zope.app.decorator o zope.app.servicenames o zope.app.timezones Alternatively, if any of those modules is not used in Zope2 (it appears that 'datetimeutls' is the only one so used, except that it uses 'timezones'), we could leave them alone. - Make the 'app' directory on the zope2 side a non-external, containing its own externals pointing to the packages we want to ship with Zope2. I don't think this should be too hard, except for the fact that I don't know how to run the functional tests on the Zope3 side: $ ../bin/python2.4 test.py -m zope.app Running unit tests: Ran 3659 tests with 0 failures and 0 errors in 52.782 seconds. Running zope.app.testing.functional.Functional tests: Set up zope.app.testing.functional.Functional Traceback (most recent call last): ... File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/src/zope/configuration/xmlconfig.py", line 394, in openInOrPlain fp = open(filename) zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/ftesting.zcml", line 20.2-20.40 IOError: [Errno 2] No such file or directory: '/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/securitypolicy.zcml' Ripping out the unused / unusable code which is sitting unused in the Zope3 tree is *not* in scope for my proposal. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEnZSz+gerLs4ltQ4RArw6AJ9LWK9/e3Roi68HDUxZppVENMKdRQCgk08A FXL+LjWC0zVpWWXclbXWzX4= =uSyp -----END PGP SIGNATURE-----
Tres Seaver wrote:
The point is that the "release" tarball should generate the same environment that the developers routinely work in; otherwise, we leave the poor suckers who install from it stuck with whatever bugs are caused by the difference.
Ok. I'll note that Python eggs don't fulfill that goal of having a development area look like a package or an installed version either.
I'll agree that these items ought not to be in the Zope2 tree: fixing that *in the checkout* would be a worthwhile effort.
Indeed; in fact, we always wished they hadn't been there in the first place. Of course, in the end we hope that zope.app won' thave to be in Zope 2 at all anymore.
The simplest approach I can see to that would be:
- On the Zope3 side, make it a rule that 'zope.app' contains only packages, not modules, converting the existing modules into packages with '__init__.py' corresponding to the current modules. Move the corresponding tests down into the new packages, as well. Currently, that would involve changing the following:
o zope.app.datetimeutils
o zope.app.decorator
o zope.app.servicenames
o zope.app.timezones
Alternatively, if any of those modules is not used in Zope2 (it appears that 'datetimeutls' is the only one so used, except that it uses 'timezones'), we could leave them alone.
- Make the 'app' directory on the zope2 side a non-external, containing its own externals pointing to the packages we want to ship with Zope2.
Sounds like the simplest approach indeed. I'm +1. I'm -1 on moving anything around other than turning modules into packages, though. Moving things around bares risks of breaking something. We can't afford that within a stable release series. Not moving the tests won't be a problem because zope.app tests aren't run in Zope 2 anyways. If they were run we'd get lots of failures anyways, because zope.app stuff is quite interwoven. Also note that all of the above have been deprecated in Zope 3.3/2.10 (datetimeutils, decorator and timezones have been moved to the zope.* level, servicenames has been made obsolete).
I don't think this should be too hard, except for the fact that I don't know how to run the functional tests on the Zope3 side:
$ ../bin/python2.4 test.py -m zope.app Running unit tests: Ran 3659 tests with 0 failures and 0 errors in 52.782 seconds. Running zope.app.testing.functional.Functional tests: Set up zope.app.testing.functional.Functional Traceback (most recent call last): ... File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/src/zope/configuration/xmlconfig.py", line 394, in openInOrPlain fp = open(filename) zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/ftesting.zcml", line 20.2-20.40 IOError: [Errno 2] No such file or directory: '/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/securitypolicy.zcml'
Looks like you forgot to run 'make' to get the ZCML stuff installed into zopeskel...?
Ripping out the unused / unusable code which is sitting unused in the Zope3 tree is *not* in scope for my proposal.
Agreed. Philipp
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Philipp von Weitershausen wrote:
Tres Seaver wrote:
The point is that the "release" tarball should generate the same environment that the developers routinely work in; otherwise, we leave the poor suckers who install from it stuck with whatever bugs are caused by the difference.
Ok. I'll note that Python eggs don't fulfill that goal of having a development area look like a package or an installed version either.
When we get there, I think the developer will install a single meta-egg, which then pulls down and installs the others. At that point, she will be working in the same environment as everyone else. When she wants to modify a package, she will check it out separately and run the 'develop' command, which will force the checkout onto her path as a "source egg". Maybe we'll even have a script available which makes this a simpler process.
I'll agree that these items ought not to be in the Zope2 tree: fixing that *in the checkout* would be a worthwhile effort.
Indeed; in fact, we always wished they hadn't been there in the first place. Of course, in the end we hope that zope.app won' thave to be in Zope 2 at all anymore.
That will be good. Even after eliminating the list you provided (see below), there is still a lot of stuff in zope.app which is irrelevant to Zope2 (I think).
The simplest approach I can see to that would be:
- On the Zope3 side, make it a rule that 'zope.app' contains only packages, not modules, converting the existing modules into packages with '__init__.py' corresponding to the current modules. Move the corresponding tests down into the new packages, as well. Currently, that would involve changing the following:
o zope.app.datetimeutils
o zope.app.decorator
o zope.app.servicenames
o zope.app.timezones
Alternatively, if any of those modules is not used in Zope2 (it appears that 'datetimeutls' is the only one so used, except that it uses 'timezones'), we could leave them alone.
- Make the 'app' directory on the zope2 side a non-external, containing its own externals pointing to the packages we want to ship with Zope2.
Sounds like the simplest approach indeed. I'm +1. I'm -1 on moving anything around other than turning modules into packages, though. Moving things around bares risks of breaking something. We can't afford that within a stable release series. Not moving the tests won't be a problem because zope.app tests aren't run in Zope 2 anyways. If they were run we'd get lots of failures anyways, because zope.app stuff is quite interwoven.
I've created a new branch from the 3.2 branch which makes the changes I proposed: i.e. 'datetimeutils', 'timezones', and 'decorators' become packages, and their tests move from 'zope/app/tests/' to their own 'tests' subdirectory (none of those tests exported facilities used by any other tests. All unit and functional tests pass on that branch, with identical results to the 3.2 tree. I have also morphed the 'zope.app' package in my 'tseaver-retire_zpkg' branch to point into this new branch, eliminating the "dead" packages you described. The only "forked" code is the empty __init__.py, which seems an acceptable tradeoff. I added an EXTERNALS.txt file which documents the intended links. BTW, a small nit of mine with subversion: can anybody tell me how to find the revision in which a directory was moved / removed, preferably via the web interface? Subversion doesn't seem to expose a way to ask about a name which, like the "man upon the stair" isn't there any more.
Also note that all of the above have been deprecated in Zope 3.3/2.10 (datetimeutils, decorator and timezones have been moved to the zope.* level, servicenames has been made obsolete).
Yup, the same general approach should work in the 2.10 tree.
I don't think this should be too hard, except for the fact that I don't know how to run the functional tests on the Zope3 side:
$ ../bin/python2.4 test.py -m zope.app Running unit tests: Ran 3659 tests with 0 failures and 0 errors in 52.782 seconds. Running zope.app.testing.functional.Functional tests: Set up zope.app.testing.functional.Functional Traceback (most recent call last): ... File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/src/zope/configuration/xmlconfig.py", line 394, in openInOrPlain fp = open(filename) zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/ftesting.zcml", line 20.2-20.40 IOError: [Errno 2] No such file or directory: '/home/tseaver/projects/Zope-CVS/Zope-3_2-branch/zopeskel/etc/securitypolicy.zcml'
Looks like you forgot to run 'make' to get the ZCML stuff installed into zopeskel...?
Hmm, I tried './configure --with-python" but there wasn't one, so I didn't even look for the Makefile. How do Zope3 developers cope with the possibility that the user may need / want to use a different Python? Running 'make PYTHON=/path/to/my/python' works, but seems a little hard-core for the make-averse crowd that hands out on #zope3-dev. ;) Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEnaym+gerLs4ltQ4RArIBAKCQSoCsS+PaAZPl4qMECiUv+fyiVQCfSVEC 3PTUmM6An/+ULklZcre8Cu0= =IYhP -----END PGP SIGNATURE-----
Tres Seaver wrote:
That will be good. Even after eliminating the list you provided (see below), there is still a lot of stuff in zope.app which is irrelevant to Zope2 (I think).
Agreed. By the way, I just compated a Zope 2.9.3 install with your tarball. The same comparison should be done again with the Zope 2.10b1 tarball as other zope.app packages might've made it into the Zope 2.10 release line. At least with explicit externals in zope.app we can avoid more zope.app junk sneaking into Zope 2 much better.
I've created a new branch from the 3.2 branch which makes the changes I proposed: i.e. 'datetimeutils', 'timezones', and 'decorators' become packages, and their tests move from 'zope/app/tests/' to their own 'tests' subdirectory (none of those tests exported facilities used by any other tests.
Ok.
I have also morphed the 'zope.app' package in my 'tseaver-retire_zpkg' branch to point into this new branch, eliminating the "dead" packages you described. The only "forked" code is the empty __init__.py, which seems an acceptable tradeoff. I added an EXTERNALS.txt file which documents the intended links.
BTW, a small nit of mine with subversion: can anybody tell me how to find the revision in which a directory was moved / removed, preferably via the web interface? Subversion doesn't seem to expose a way to ask about a name which, like the "man upon the stair" isn't there any more.
I usually google to find the checkin email :).
Hmm, I tried './configure --with-python" but there wasn't one
Right, it's a checkout. What's there to configure? (Besides the Python executable).
so I didn't even look for the Makefile. How do Zope3 developers cope with the possibility that the user may need / want to use a different Python? Running 'make PYTHON=/path/to/my/python' works, but seems a little hard-core for the make-averse crowd that hands out on #zope3-dev. ;)
Heh, but that's how we do it :) Philipp
Tres Seaver wrote:
BTW, a small nit of mine with subversion: can anybody tell me how to find the revision in which a directory was moved / removed, preferably via the web interface? Subversion doesn't seem to expose a way to ask about a name which, like the "man upon the stair" isn't there any more.
Something based on svn log -v, for instance: svn log -v | awk '/^(r| D}/{print};' Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
Philipp von Weitershausen wrote:
Tres Seaver wrote:
The point is that the "release" tarball should generate the same environment that the developers routinely work in; otherwise, we leave the poor suckers who install from it stuck with whatever bugs are caused by the difference. Ok. I'll note that Python eggs don't fulfill that goal of having a development area look like a package or an installed version either.
When we get there, I think the developer will install a single meta-egg, which then pulls down and installs the others. At that point, she will be working in the same environment as everyone else. When she wants to modify a package, she will check it out separately and run the 'develop' command, which will force the checkout onto her path as a "source egg". Maybe we'll even have a script available which makes this a simpler process.
I'll agree that these items ought not to be in the Zope2 tree: fixing that *in the checkout* would be a worthwhile effort. Indeed; in fact, we always wished they hadn't been there in the first place. Of course, in the end we hope that zope.app won' thave to be in Zope 2 at all anymore.
That will be good. Even after eliminating the list you provided (see below), there is still a lot of stuff in zope.app which is irrelevant to Zope2 (I think).
The simplest approach I can see to that would be:
- On the Zope3 side, make it a rule that 'zope.app' contains only packages, not modules, converting the existing modules into packages with '__init__.py' corresponding to the current modules. Move the corresponding tests down into the new packages, as well. Currently, that would involve changing the following:
o zope.app.datetimeutils
o zope.app.decorator
o zope.app.servicenames
o zope.app.timezones
Alternatively, if any of those modules is not used in Zope2 (it appears that 'datetimeutls' is the only one so used, except that it uses 'timezones'), we could leave them alone.
- Make the 'app' directory on the zope2 side a non-external, containing its own externals pointing to the packages we want to ship with Zope2. Sounds like the simplest approach indeed. I'm +1. I'm -1 on moving anything around other than turning modules into packages, though. Moving things around bares risks of breaking something. We can't afford that within a stable release series. Not moving the tests won't be a problem because zope.app tests aren't run in Zope 2 anyways. If they were run we'd get lots of failures anyways, because zope.app stuff is quite interwoven.
I've created a new branch from the 3.2 branch which makes the changes I proposed: i.e. 'datetimeutils', 'timezones', and 'decorators' become packages, and their tests move from 'zope/app/tests/' to their own 'tests' subdirectory (none of those tests exported facilities used by any other tests.
All unit and functional tests pass on that branch, with identical results to the 3.2 tree.
I have also morphed the 'zope.app' package in my 'tseaver-retire_zpkg' branch to point into this new branch, eliminating the "dead" packages you described. The only "forked" code is the empty __init__.py, which seems an acceptable tradeoff. I added an EXTERNALS.txt file which documents the intended links.
BTW, a small nit of mine with subversion: can anybody tell me how to find the revision in which a directory was moved / removed, preferably via the web interface? Subversion doesn't seem to expose a way to ask about a name which, like the "man upon the stair" isn't there any more.
Also note that all of the above have been deprecated in Zope 3.3/2.10 (datetimeutils, decorator and timezones have been moved to the zope.* level, servicenames has been made obsolete).
Yup, the same general approach should work in the 2.10 tree.
I've now done the same work on a branch for that tree: [/home/tseaver/projects/Zope-CVS/tseaver-retire_zpkg-2.10] $ head .svn/entries <?xml version="1.0" encoding="utf-8"?> <wc-entries xmlns="svn:"> <entry committed-rev="68857" name="" committed-date="2006-06-26T21:16:14.182239Z" url="svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg-2.10" last-author="tseaver" kind="dir" Note that here I used the contents of 'zope.app' as installed by the 2.10b1 tarball, which has at least one of the packages in it ('wfmc') which Philipp said should be excluded for 2.9. AFAIK, both of these branches are ready for merging to their respective branches (and to the trunk, for the 2.10 version). Comments? Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEoFIz+gerLs4ltQ4RApTFAKCRpYarfrxQcBkZAaTmFqYWemQ6ZQCghVpS aYhIXCIMVqta0V+LH9gD7Tc= =V2vf -----END PGP SIGNATURE-----
Tres Seaver wrote:
I've now done the same work on a branch for that tree:
[/home/tseaver/projects/Zope-CVS/tseaver-retire_zpkg-2.10] $ head .svn/entries <?xml version="1.0" encoding="utf-8"?> <wc-entries xmlns="svn:"> <entry committed-rev="68857" name="" committed-date="2006-06-26T21:16:14.182239Z"
url="svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg-2.10" last-author="tseaver" kind="dir"
Btw, I found 'svn info' the other day. Before that I was looking at .svn/entries, too.
Note that here I used the contents of 'zope.app' as installed by the 2.10b1 tarball, which has at least one of the packages in it ('wfmc') which Philipp said should be excluded for 2.9.
AFAIK, both of these branches are ready for merging to their respective branches (and to the trunk, for the 2.10 version).
Comments?
Looks good. Let's merge :) Philipp
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Philipp von Weitershausen wrote:
Tres Seaver wrote:
I've now done the same work on a branch for that tree:
[/home/tseaver/projects/Zope-CVS/tseaver-retire_zpkg-2.10] $ head .svn/entries <?xml version="1.0" encoding="utf-8"?> <wc-entries xmlns="svn:"> <entry committed-rev="68857" name="" committed-date="2006-06-26T21:16:14.182239Z"
url="svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg-2.10" last-author="tseaver" kind="dir"
Btw, I found 'svn info' the other day. Before that I was looking at .svn/entries, too.
Note that here I used the contents of 'zope.app' as installed by the 2.10b1 tarball, which has at least one of the packages in it ('wfmc') which Philipp said should be excluded for 2.9.
AFAIK, both of these branches are ready for merging to their respective branches (and to the trunk, for the 2.10 version).
Comments?
Looks good. Let's merge :)
I'm working on the merge right now, and having some problems: $ export ZSVN=svn+ssh://svn.zope.org/repos/main $ svn co $ZSVN/Zope/branches/2.9 Zope-2_9-branch ... $ cd Zope-2_9-branch $ svn merge -r 68803:HEAD $ZSVN/Zope/branches/tseaver-retire_zpkg . D releases/Zope2/DEPENDENCIES.cfg D releases/Zope2/test.py D releases/Zope2/PUBLICATION.cfg D releases/Zope2/PACKAGE.cfg D releases/Zope2/zopetest D releases/Zope2/SETUP.cfg D releases/Zope2 D releases/Zope2.map D releases/Zope2.cfg D releases Performing status on external item at 'buildsupport/zpkgsetup' Skipped 'buildsupport' U configure U doc/CHANGES.txt U lib/python/Zope2 U lib/python/Products/Transience/Transience.py U lib/python/Products/ZCatalog/ProgressHandler.py U lib/python/Products/ZCatalog/ZCatalog.py U lib/python/Products/PluginIndexes/DateIndex/DateIndex.py U lib/python/Products/PluginIndexes/TextIndex/TextIndex.py U lib/python/App/RefreshFuncs.py U lib/python/App/ProductContext.py U lib/python/OFS/Cache.py U lib/python/OFS/ObjectManager.py U lib/python/tempstorage/TemporaryStorage.py U lib/python/Shared/DC/ZRDB/Connection.py svn: URL 'svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg/lib/python/zope/app' doesn't match existing URL 'svn://svn.zope.org/repos/main/Zope3/tags/Zope-3.2.1/src/zope/app' in 'lib/python/zope/app' I may have to munge 'zope/app' manually and check it in to get the merge done. Anybody have another suggestion? Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEq9gh+gerLs4ltQ4RAlEKAJ49BdccSaBRoPtcbHlDGc+gLol2GwCdF6Eu IT+gW5v9CNOrLPiNa9fWjxg= =C6CD -----END PGP SIGNATURE-----
Tres Seaver wrote:
svn: URL 'svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg/lib/python/zope/app' doesn't match existing URL 'svn://svn.zope.org/repos/main/Zope3/tags/Zope-3.2.1/src/zope/app' in 'lib/python/zope/app'
I may have to munge 'zope/app' manually and check it in to get the merge done. Anybody have another suggestion?
When replacing a subversioned directory with an external, you first need to remove the directory and commit it. Then, in a next revision, you add the external. Philipp
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Philipp von Weitershausen wrote:
Tres Seaver wrote:
svn: URL 'svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg/lib/python/zope/app' doesn't match existing URL 'svn://svn.zope.org/repos/main/Zope3/tags/Zope-3.2.1/src/zope/app' in 'lib/python/zope/app'
I may have to munge 'zope/app' manually and check it in to get the merge done. Anybody have another suggestion?
When replacing a subversioned directory with an external, you first need to remove the directory and commit it. Then, in a next revision, you add the external.
I'm actually going the other direction, but I think I still need two transactions (first remove the external, then create and add tne directory) Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFErHJv+gerLs4ltQ4RAq24AKC1aNL4mSW7UO71KnwXgT8U/ZYtbACeLUGz 4cMFJMBosBpvuiuB9qFhwZ8= =3uV7 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tres Seaver wrote:
Philipp von Weitershausen wrote:
Tres Seaver wrote:
svn: URL 'svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg/lib/python/zope/app' doesn't match existing URL 'svn://svn.zope.org/repos/main/Zope3/tags/Zope-3.2.1/src/zope/app' in 'lib/python/zope/app'
I may have to munge 'zope/app' manually and check it in to get the merge done. Anybody have another suggestion? When replacing a subversioned directory with an external, you first need to remove the directory and commit it. Then, in a next revision, you add the external.
I'm actually going the other direction, but I think I still need two transactions (first remove the external, then create and add tne directory)
OK, I have merged the 'tseaver-retire_zpkg_branch' to the 2.9 branch, and the 'tseaver-zpkg_retire-2.10' branch to the 2.10 branch and the trunk. Because of the surgery involved in switching 'lib/python/zope/app' from an svn:external to a "real" directlry, your existing sandboxen may neeed some help in completing an 'svn up' (try removing the app directory, if it barfs there). Note that I did *not* repair the test failure which showed up on the 2.10 branch / trunk while I was in the middle of the merge -- it looks as though it might be related to the traversal changes Florent made today, and the failure is the same on the 2.10 branch / trunk *without* the merge as *with* it. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFErI5h+gerLs4ltQ4RAiMeAJwMcaKkQRt7/V4kf2aoKL6wBjrTpgCYgVHd fzEB9F5epsoQ83iUE/PR3w== =UfN3 -----END PGP SIGNATURE-----
Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Tres Seaver wrote:
Philipp von Weitershausen wrote:
Tres Seaver wrote:
svn: URL 'svn+ssh://svn.zope.org/repos/main/Zope/branches/tseaver-retire_zpkg/lib/python/zope/app' doesn't match existing URL 'svn://svn.zope.org/repos/main/Zope3/tags/Zope-3.2.1/src/zope/app' in 'lib/python/zope/app'
I may have to munge 'zope/app' manually and check it in to get the merge done. Anybody have another suggestion? When replacing a subversioned directory with an external, you first need to remove the directory and commit it. Then, in a next revision, you add the external. I'm actually going the other direction, but I think I still need two transactions (first remove the external, then create and add tne directory)
OK, I have merged the 'tseaver-retire_zpkg_branch' to the 2.9 branch, and the 'tseaver-zpkg_retire-2.10' branch to the 2.10 branch and the trunk.
Because of the surgery involved in switching 'lib/python/zope/app' from an svn:external to a "real" directlry, your existing sandboxen may neeed some help in completing an 'svn up' (try removing the app directory, if it barfs there).
Note that I did *not* repair the test failure which showed up on the 2.10 branch / trunk while I was in the middle of the merge -- it looks as though it might be related to the traversal changes Florent made today, and the failure is the same on the 2.10 branch / trunk *without* the merge as *with* it.
Yep I'll check and fix those asap. Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
Philipp von Weitershausen wrote at 2006-6-24 20:03 +0200:
... c) are so much core to Zope 3 that they don't make sense in Zope 2 (such as twisted, securitypolicy)
When I remember right, we should get "twisted" in Zope 2.10? -- Dieter
Dieter Maurer wrote:
Philipp von Weitershausen wrote at 2006-6-24 20:03 +0200:
... c) are so much core to Zope 3 that they don't make sense in Zope 2 (such as twisted, securitypolicy)
When I remember right, we should get "twisted" in Zope 2.10?
My mentioning of 'twisted' above actually refers to zope.app.twisted (the context was packages from zope.app). AFAIK the ZPublisher has been amended with experimental twisted support, but that doesn't mean we have to ship twisted with Zope 2.10. In fact, Zope 2.10b1 doesn't ship with it and I don't think we should change anything there. Philipp
participants (11)
-
Andreas Jung -
Chris McDonough -
Dieter Maurer -
Florent Guillaume -
Fred Drake -
Jens Vagelpohl -
Lennart Regebro -
Martijn Faassen -
Philipp von Weitershausen -
Sidnei da Silva -
Tres Seaver