Hi everybody, The Grok team is happy to announce the release of Grok 0.11! Grok is a web application framework built with Zope 3 technologies that aims to be easy to use. Grok 0.11 is a feature release of Grok, introducing important new features: * Layers and skinning support. It is now easy to place views in layers, and to define skins so that these layers can be browsed. * REST support. There is an easy way to create RESTful web services with Grok now. Create special REST protocols that support GET, POST, PUT and DELETE. See http://grok.zope.org/minitutorials/rest.html for more information. * pluggable template languages. Besides the built-in support for Zope Page Templates, Grok now allows you to plug in other template languages as well. See http://grok.zope.org/minitutorials/template-languages.html for instructions on how to integrate a template language into Grok. We are planning on releasing a Genshi plugin for Grok next week. * Improved permission and role support. See http://grok.zope.org/minitutorials/permissions.html for more information. If you were already using these in your existing Grok code you need to change your code a bit. See http://grok.zope.org/upgrade.html contains more information about these changes. One important change is rather hidden from view so bears a bit of explanation. Grok is based on Zope 3. Grok could always reuse Zope 3 libraries without problems. The other way around wasn't so nice however: Zope 3-based code could not use Grok-based code so easily, as Grok would not emit configuration actions. Grok 0.11 now does emit such configuration actions and this is a major step on the way towards backwards compatibility with Zope 3. If you have custom Grokkers in your code, you probably have to change them: please see http://grok.zope.org/upgrade.txt for information how to upgrade these. For the detailed changelog, see Grok's PyPI entry page: http://pypi.python.org/pypi/grok For upgrade notes, including how to change your application, see http://grok.zope.org/upgrade.html For installation instructions and much more on how to use Grok, see the Grok tutorial: http://grok.zope.org/tutorial.html If you enjoy Grok, please subscribe to the grok-dev mailing list http://mail.zope.org/mailman/listinfo/grok-dev and join us in the #grok channel on irc.freenode.net We hope to see you there!
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote:
Hi everybody,
The Grok team is happy to announce the release of Grok 0.11! Grok is a web application framework built with Zope 3 technologies that aims to be easy to use.
Grok 0.11 is a feature release of Grok, introducing important new features:
* Layers and skinning support. It is now easy to place views in layers, and to define skins so that these layers can be browsed.
* REST support. There is an easy way to create RESTful web services with Grok now. Create special REST protocols that support GET, POST, PUT and DELETE. See http://grok.zope.org/minitutorials/rest.html for more information.
* pluggable template languages. Besides the built-in support for Zope Page Templates, Grok now allows you to plug in other template languages as well. See http://grok.zope.org/minitutorials/template-languages.html for instructions on how to integrate a template language into Grok. We are planning on releasing a Genshi plugin for Grok next week.
* Improved permission and role support. See http://grok.zope.org/minitutorials/permissions.html for more information. If you were already using these in your existing Grok code you need to change your code a bit. See http://grok.zope.org/upgrade.html contains more information about these changes.
One important change is rather hidden from view so bears a bit of explanation. Grok is based on Zope 3. Grok could always reuse Zope 3 libraries without problems. The other way around wasn't so nice however: Zope 3-based code could not use Grok-based code so easily, as Grok would not emit configuration actions. Grok 0.11 now does emit such configuration actions and this is a major step on the way towards backwards compatibility with Zope 3. If you have custom Grokkers in your code, you probably have to change them: please see http://grok.zope.org/upgrade.txt for information how to upgrade these.
For the detailed changelog, see Grok's PyPI entry page: http://pypi.python.org/pypi/grok
For upgrade notes, including how to change your application, see http://grok.zope.org/upgrade.html
For installation instructions and much more on how to use Grok, see the Grok tutorial: http://grok.zope.org/tutorial.html
If you enjoy Grok, please subscribe to the grok-dev mailing list
http://mail.zope.org/mailman/listinfo/grok-dev
and join us in the #grok channel on irc.freenode.net
We hope to see you there!
The new grok-0.11.cfg adds dependencies on eggs which should *not* exist: $ diff -u configs/grok-0.1{0.2,1}.cfg - --- configs/grok-0.10.2.cfg 2007-10-29 18:32:29.000000000 -0400 +++ configs/grok-0.11.cfg 2007-11-08 19:44:06.000000000 -0500 @@ -1,8 +1,8 @@ [versions] - -grok = 0.10.2 +grok = 0.11 ClientForm = 0.2.7 docutils = 0.4 - -martian = 0.9 +martian = 0.9.1 mechanize = 0.1.7b Pygments = 0.8.1 pytz = 2007g @@ -44,7 +44,7 @@ zope.app.preference = 3.4.0a1 zope.app.principalannotation = 3.4.0a1 zope.app.publication = 3.4.2 - -zope.app.publisher = 3.4.0 +zope.app.publisher = 3.5.0a2 zope.app.renderer = 3.4.0a1 zope.app.rotterdam = 3.4.0a1 zope.app.schema = 3.4.0a1 @@ -86,16 +86,16 @@ zope.modulealias = 3.4.0a1 zope.pagetemplate = 3.4.0a1 zope.proxy = 3.4.0 - -zope.publisher = 3.4.1 +zope.publisher = 3.5.0a1.dev-r78838 zope.schema = 3.4.0 zope.security = 3.4.0b5 - -zope.server = 3.4.1 +zope.server = 3.5.0a2 zope.session = 3.4.1 zope.size = 3.4.0 zope.structuredtext = 3.4.0 zope.tal = 3.4.0b1 zope.tales = 3.4.0a1 zope.testbrowser = 3.4.1 - -zope.testing = 3.4 +zope.testing = 3.5.1 zope.thread = 3.4 - -zope.traversing = 3.4.0 +zope.traversing = 3.5.0a1.dev-r78730 Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them. DEATH TO FAUX PACKAGES! Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHM7Gd+gerLs4ltQ4RAumxAJ9wIYGXkQIhNHqmktgu05U8HY9PYgCgoSph tLjLyiWXmmR4982xesFWgd0= =25Rh -----END PGP SIGNATURE-----
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES!
As far as I understand, this does not happen if you depend on a KGS, right? Does the grok release not use the KGS from Stephan? Regards Roger Ineichen
Tres.
On Friday 09 November 2007, Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES!
As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
I agree with Roger, Grok should use the KGS and all will be fine. ;-) Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Stephan Richter wrote:
On Friday 09 November 2007, Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES! As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
I agree with Roger, Grok should use the KGS and all will be fine. ;-)
It can't because you guys broke a dozen packages when you split up some of the zope.app.* packages. I'm going to fix this now. Also, I have a couple of beefs with the "locked down index" approach to the KGS. For instance, if we use the Zope 3.4.x KGS, how would we lock down the version of the grok egg? Or simplejson, z3c.flashmessage and all the other Grok-specific dependencies? Create our own locked down index? I don't think so.
On Friday 09 November 2007, Philipp von Weitershausen wrote:
It can't because you guys broke a dozen packages when you split up some of the zope.app.* packages. I'm going to fix this now.
Can we stop pointing fingers, please? I just spent 3-4 man-weeks to get the packages to a decent state, produce the KGS and write tools to make the entire process a little saner. Certainly not a mess that I started.
Also, I have a couple of beefs with the "locked down index" approach to the KGS. For instance, if we use the Zope 3.4.x KGS, how would we lock down the version of the grok egg? Or simplejson, z3c.flashmessage and all the other Grok-specific dependencies? Create our own locked down index? I don't think so.
For all packages living at svn.zope.org, the KGS is open and people can submit additions, as I offered in a recent zope-dev mail. (I wonder whether anyone read it.) As for other packages, you have two choices: (1) Fix the versions in the [version] section of buildout. (2) Derive a new KGS based on the Zope 3.4 one. When Jim and I discussed the KGS and how to implement it initially, we had the second solution in mind. In fact, we talked in particular about grok as an example. Implementing support for (2) should be pretty simple. You simply have to enhance the zc.mirrorcheeseshopslashsimple code to accept multiple controlled-pages.cfg files. This should take about 30 mins. If you want some of the testing comforts that I have enjoyed, then you need to adjust zope.release to handle multiple controlled-packages.cfg files as well. This is an overall easy task and getting the index online is also pretty straightforward. I use the following crontab configuration: MAILTO=stephan.richter@gmail.com * * * * * /home/srichter/mirror-tool/bin/generate-controlled-pages /var/www/download.zope.org/zop e3.4 * * * * * /home/srichter/mirror-tool/bin/update-simple-mirror /var/www/download.zope.org/zope3.4 Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Stephan Richter wrote:
On Friday 09 November 2007, Philipp von Weitershausen wrote:
It can't because you guys broke a dozen packages when you split up some of the zope.app.* packages. I'm going to fix this now.
Can we stop pointing fingers, please?
Sorry for not sugar coating the facts. I gladly will stop pointing fingers when you guys fix the bugs that you introduced and have been made aware of, e.g. https://bugs.launchpad.net/zope3/+bug/158743. Not to worry now, I finally found the time to do it myself at the Boring Sprint now.
I just spent 3-4 man-weeks to get the packages to a decent state, produce the KGS and write tools to make the entire process a little saner. Certainly not a mess that I started.
Also, I have a couple of beefs with the "locked down index" approach to the KGS. For instance, if we use the Zope 3.4.x KGS, how would we lock down the version of the grok egg? Or simplejson, z3c.flashmessage and all the other Grok-specific dependencies? Create our own locked down index? I don't think so.
For all packages living at svn.zope.org, the KGS is open and people can submit additions, as I offered in a recent zope-dev mail. (I wonder whether anyone read it.)
As for other packages, you have two choices:
(1) Fix the versions in the [version] section of buildout.
(2) Derive a new KGS based on the Zope 3.4 one.
When Jim and I discussed the KGS and how to implement it initially, we had the second solution in mind. In fact, we talked in particular about grok as an example.
Implementing support for (2) should be pretty simple. You simply have to enhance the zc.mirrorcheeseshopslashsimple code to accept multiple controlled-pages.cfg files. This should take about 30 mins.
The versions.cfg file loaded from HTTP works now.
If you want some of the testing comforts that I have enjoyed, then you need to adjust zope.release to handle multiple controlled-packages.cfg files as well. This is an overall easy task and getting the index online is also pretty straightforward. I use the following crontab configuration:
MAILTO=stephan.richter@gmail.com * * * * * /home/srichter/mirror-tool/bin/generate-controlled-pages /var/www/download.zope.org/zop e3.4 * * * * * /home/srichter/mirror-tool/bin/update-simple-mirror /var/www/download.zope.org/zope3.4
So you're telling me that in order to define my own KGS (which anybody should do for a serious project), I'll have to start deploying my own index. For each project?
Philipp von Weitershausen wrote: [snip]
So you're telling me that in order to define my own KGS (which anybody should do for a serious project), I'll have to start deploying my own index. For each project?
I think Grok should be using KGS by simply taking snapshots of it sometimes and then building an own list of absolutely fixed versions from it. There are a lot of problems not solved yet concerning fixing versions in any solution I've seen so far. Grok can fix a single list right now, but what if we release megrok.genshi which relies on a particular version of Genshi? If you are using that in your application, you'll have to add it and all its dependencies to buildout.cfg manually, *or* megrok.genshi will have to release its own versions.cfg somewhere on some website that we then also ask people to extend. Anyway, all this requires quite a bit of overhead and typing. If we relied on KGS, we could release a fixed version of megrok.genshi, but what to do when it needs an update? We don't want to force all users to update - this needs to be under the application developer's control. KGS is good for integration tests and developing against, but I don't see it as a good solution for releasing against. I'm still convinced the real solution involves storing fixed version number requirements where they belong: in the packages themselves (with an 'or' story in setup.py). This isn't perfect either, as the packages may disappear from the cheeseshop, so a separate index like KGS would be desirable in combination with this approach. I want a solution where someone can say: "I want to rely on foo 0.7", and then it'll get that and a fixed version of all its dependencies (unless it's explicitly overridden). I want people to be able to say this with a minimal amount of work (no need to set up and maintain your own index) and no risk at all that later on suddenly you'll be getting different versions. Not even bugfixes. Regards, Martijn
Hi Philipp
Betreff: Re: AW: Re: Grok 0.11 released!
Stephan Richter wrote:
On Friday 09 November 2007, Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES! As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
I agree with Roger, Grok should use the KGS and all will be fine. ;-)
It can't because you guys broke a dozen packages when you split up some of the zope.app.* packages. I'm going to fix this now.
Also, I have a couple of beefs with the "locked down index" approach to the KGS. For instance, if we use the Zope 3.4.x KGS, how would we lock down the version of the grok egg? Or simplejson, z3c.flashmessage and all the other Grok-specific dependencies? Create our own locked down index? I don't think so.
I think so because, Before KGS there was no concept for keep the eggs in sync. Each project has to do it by itself. KGS at least is something that helps to manage the zope eggs. Everything outside this has to be done by the release manager of the project. And before the KGS, the cheesshop was nothing else then a different concept for get packages. The reason why is; Fixing revision numbers in packages does not work!!! Fixing version will make it impossible to use it as base and mix other packge into a setup. Because of the potential conflicting packages you will mixin. e.g. if a package foo-0.1 use bar-0.1 and bar get update to bar-0.2 and breaks, you can update bar to bar-0.3 and make it work again. If you fix the version in package foo-0.1 only to use bar-0.1 because the broken bar-0.2 you will get in trouble. Because foo-0.1 works with bar-0.1 and bar-0.3. The way to go is: You can use a KGS and define that you use foo-0.1 and bar-0.2 or foo-0.1 and bar-0.2. Agan, if you try to fix this in the egg itself, you get blocked sooner or later. I think the next step for KGS is to define more KGS packages. This will make sure that we have a better controll of what is working with what. Note, we lost all this since the eggs do not have a realation to the subsversion version numbers. The Subversion did ensure that for a specific timestamp everything (in the trunk) was working with each other. That's history and now a part of the release process. But the good thing about eggs and the KGS now is, that we are able to define the working set for packages outside the Zope trunk too. Regards Roger Ineichen
On Nov 9, 2007 4:06 PM, Roger Ineichen <dev@projekt01.ch> wrote:
Create our own locked down index?
I think so because,
Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all". Because that's what we all need. So it must be possible, right? The KGS is just that, a Known Good Set. It must be possible to in your build out override a version here and add packages there? -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
On Friday 09 November 2007, Lennart Regebro wrote:
On Nov 9, 2007 4:06 PM, Roger Ineichen <dev@projekt01.ch> wrote:
Create our own locked down index?
I think so because,
Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all".
Unfortunately, this is not how the index works in setuptools.
Because that's what we all need. So it must be possible, right? The KGS is just that, a Known Good Set. It must be possible to in your build out override a version here and add packages there?
So the way to do this is to use versions.cfg from the KGS, which locks to the latest version of all packages in the KGS; since the URL can be pointed directly to the KGS server, you will get all the updates too. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Stephan Richter wrote:
On Friday 09 November 2007, Lennart Regebro wrote:
On Nov 9, 2007 4:06 PM, Roger Ineichen <dev@projekt01.ch> wrote:
Create our own locked down index? I think so because, Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all".
Unfortunately, this is not how the index works in setuptools.
Which is why I think the index-page KGS solution is largely unusable for the projects I work in. :-/ Can you please also confirm something for me - if I use the Zope KGS index page, and I also need a package from the Cheese Shop, and the Cheese Shop contains newer (not-good) versions of packages that are also in the KGS index, will setuptools/buildout use the one from the KGS without a narrow version spec in my egg dependencies? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Friday 09 November 2007, Martin Aspeli wrote:
Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all".
Unfortunately, this is not how the index works in setuptools.
Which is why I think the index-page KGS solution is largely unusable for the projects I work in. :-/
See my new response to Lennart.
Can you please also confirm something for me - if I use the Zope KGS index page, and I also need a package from the Cheese Shop, and the Cheese Shop contains newer (not-good) versions of packages that are also in the KGS index, will setuptools/buildout use the one from the KGS without a narrow version spec in my egg dependencies?
The KGS publishes all versions of all PyPI packages that are not controlled. That index is updated every minute. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Lennart Regebro wrote:
On Nov 9, 2007 4:06 PM, Roger Ineichen <dev@projekt01.ch> wrote:
Create our own locked down index? I think so because,
Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all".
Because that's what we all need. So it must be possible, right? The KGS is just that, a Known Good Set. It must be possible to in your build out override a version here and add packages there?
Those are exactly the problems with the locked down index: * You can always use *more* packages but you'll have to lock them down yourself, can't use another KGS for those * You cannot say "I know the KGS prefers zope.frobnaz 1.2.3, but I deliberately want to upgrade to zope.frobnaz 2.0.2". It won't work because the index simply doesn't expose the 2.0.2 version of zope.frobnaz. So the "locked down index" doesn't support two important use cases. And building your own index isn't really a solution, IMO. The average cavemen programmer won't do that...
On Friday 09 November 2007, Philipp von Weitershausen wrote:
Those are exactly the problems with the locked down index:
* You can always use *more* packages but you'll have to lock them down yourself, can't use another KGS for those
But this is a limitation of setuptools. I think this could be solved using find-links for now.
* You cannot say "I know the KGS prefers zope.frobnaz 1.2.3, but I deliberately want to upgrade to zope.frobnaz 2.0.2". It won't work because the index simply doesn't expose the 2.0.2 version of zope.frobnaz.
You can use find-links and point to the new location.
So the "locked down index" doesn't support two important use cases. And building your own index isn't really a solution, IMO. The average cavemen programmer won't do that...
The problem here is that setuptools is too immature to handle those cases properly. We will have to write a lot more software to do this correctly. The KGS index approach was just something that would give us sanity with very little new software. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Nov 9, 2007 4:34 PM, Philipp von Weitershausen <philipp@weitershausen.de> wrote:
Those are exactly the problems with the locked down index:
* You can always use *more* packages but you'll have to lock them down yourself, can't use another KGS for those
Could we create a grok KGS which is generated by a python-script, that takes the correct zope3 KGS as a base?
* You cannot say "I know the KGS prefers zope.frobnaz 1.2.3, but I deliberately want to upgrade to zope.frobnaz 2.0.2". It won't work because the index simply doesn't expose the 2.0.2 version of zope.frobnaz.
This needs to be fixed in setup tools some time in the future, then, because it will severly limit the usablility of a KGSs if you can't use this. You very often need to upgrade one package... -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
On Friday 09 November 2007, Lennart Regebro wrote:
On Nov 9, 2007 4:34 PM, Philipp von Weitershausen
<philipp@weitershausen.de> wrote:
Those are exactly the problems with the locked down index:
* You can always use *more* packages but you'll have to lock them down yourself, can't use another KGS for those
Could we create a grok KGS which is generated by a python-script, that takes the correct zope3 KGS as a base?
With a little bit more software, yes.
* You cannot say "I know the KGS prefers zope.frobnaz 1.2.3, but I deliberately want to upgrade to zope.frobnaz 2.0.2". It won't work because the index simply doesn't expose the 2.0.2 version of zope.frobnaz.
This needs to be fixed in setup tools some time in the future, then, because it will severly limit the usablility of a KGSs if you can't use this. Â You very often need to upgrade one package...
Right, I agree. But there are solutions. You can use find-links to get a newer version. The beautiful thing about not nailing versions is that you can overwrite what the index says. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Friday 09 November 2007, Lennart Regebro wrote:
On Nov 9, 2007 4:34 PM, Philipp von Weitershausen
<philipp@weitershausen.de> wrote:
Those are exactly the problems with the locked down index:
* You can always use *more* packages but you'll have to lock them down yourself, can't use another KGS for those
Could we create a grok KGS which is generated by a python-script, that takes the correct zope3 KGS as a base?
Based on my son's behavior today, you will be able to do this in a few hours.
* You cannot say "I know the KGS prefers zope.frobnaz 1.2.3, but I deliberately want to upgrade to zope.frobnaz 2.0.2". It won't work because the index simply doesn't expose the 2.0.2 version of zope.frobnaz.
With the extends/bases behavior I am planning to add, you will be able to do exactly this.
This needs to be fixed in setup tools some time in the future, then, because it will severly limit the usablility of a KGSs if you can't use this. Â You very often need to upgrade one package...
I think that with a little bit more code for the KGS and considering all the other possibilities, such as find-links and versions, we have enough flexibility. I think the challenge will be to come up with a recommended way to do all these things. Maybe we should collect our use cases again and see how they can be solved. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Saturday 10 November 2007, Stephan Richter wrote:
Could we create a grok KGS which is generated by a python-script, that takes the correct zope3 KGS as a base?
Based on my son's behavior today, you will be able to do this in a few hours.
Okay, this is now done. It is all documented here: http://svn.zope.org/zope.release/trunk/src/zope/release/README.txt?view=mark... Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Friday 09 November 2007, Lennart Regebro wrote:
On Nov 9, 2007 4:06 PM, Roger Ineichen <dev@projekt01.ch> wrote:
Create our own locked down index?
I think so because,
Surely there must be a way to say "I want to use THAT KGS over there. Except for this module, that should hav this version. And of course, all these bunch of eggs which isn't dfined in that KGS at all".
Sure, now that I think of it. ;-) [buildout] index = http://download.zope.org/zope3.4 find-links = \ http://pypi.python.org/packages/source/z/zope.component/zope.component-3.4.0...
Because that's what we all need. So it must be possible, right? The KGS is just that, a Known Good Set. It must be possible to in your build out override a version here and add packages there?
Yep. I will also publish an extension to the KGS files today that allows you to manage your own ``controlled-packages.cfg`` file based on Zope 3.4's; but you will not need to have your own KGS, since you can use the ``controlled-packages.cfg`` file to generate a ``versions.cfg`` file. Mmmh, the more I think about it, the more I become convinced that we are getting to a pretty solid story here. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Roger Ineichen wrote:
Before KGS there was no concept for keep the eggs in sync. Each project has to do it by itself. KGS at least is something that helps to manage the zope eggs. Everything outside this has to be done by the release manager of the project.
And before the KGS, the cheesshop was nothing else then a different concept for get packages.
The reason why is; Fixing revision numbers in packages does not work!!!
Fixing version will make it impossible to use it as base and mix other packge into a setup. Because of the potential conflicting packages you will mixin.
I think we're now quite aware of that problem. Hence we need the KGS. But I'm questioning the implementation using a "locked down index".
On Friday 09 November 2007, Philipp von Weitershausen wrote:
I think we're now quite aware of that problem. Hence we need the KGS. But I'm questioning the implementation using a "locked down index".
What's the alternative? Nailing versions is not it for me, which by the way you can do with http://download.zope.org/Zope3.4/versions.cfg already. Can you specify multiple version URLs? Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES!
As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
Not yet. We intend to look at that list at some point. Of course the KGS probably doesn't contain recent enough packages to support the REST changes. The other problem with KGS is that I absolutely want to fix lists of packages into Grok itself and never rely on any system that can cause the list of packages that could be updated. KGS will get bugfix releases. These will inevitably break something on occassion. I have no idea what will happen once 3.5 packages will start to appear, either. Regarsd, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martijn Faassen wrote:
Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES! As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
Not yet. We intend to look at that list at some point. Of course the KGS probably doesn't contain recent enough packages to support the REST changes.
If the REST stuff depended on unreleased versions of upstream packages, then it should not have been merged until those packages were released in an acceptable (non-snapshot) form.
The other problem with KGS is that I absolutely want to fix lists of packages into Grok itself and never rely on any system that can cause the list of packages that could be updated. KGS will get bugfix releases. These will inevitably break something on occassion. I have no idea what will happen once 3.5 packages will start to appear, either.
A KGS needs to have the following properties: - The "generation zero" of any KGS is an empty set of revisions. - By default, any revision N of a KGS starts out as a "draft" version which is an empty layer over version N-1. Changes to the draft then shadow any versions in the prior revision. - Once "finalized" / "published", a given revision of a KGS can *never* be changed. - Any KGS can be derived from the published version of another KGS, with additions of new distributions / versions and updates of versions for underlying distributions. - In conjunction with a "find-links" site which promises never to remove any distribution which has been included in a published revision of a KGS, the current 'version.cfg' (and workalike) files are sufficient to establish a KGS. Without that promise, however, those files can't be considered as defining a KGS. Therefore, - Given that all distribution versions mentioned in a KGS are stored at a trusted / reliable URL, any immutable KGS revision can be trivially transformed into a PyPI package index: each distribution will have exactly one allowed version, which will point to a single URL on a reliable server. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHNS+v+gerLs4ltQ4RAl/rAJ9lDeB9nOXGab+/uAithNjeMeREkwCgguHF 5qA0vftURPPCW7SR2yZVTmk= =NBMc -----END PGP SIGNATURE-----
On Nov 10, 2007 5:12 AM, Tres Seaver <tseaver@palladion.com> wrote:
If the REST stuff depended on unreleased versions of upstream packages, then it should not have been merged until those packages were released in an acceptable (non-snapshot) form.
That's a good sentiment and I agre with it in practice. But Grok is still in < 1.0 status so I don't eally think it's a big issue. Waiting for the zope packages to stabilize would only have delayed the progress of Grok at this moment, and the amount of people using Grok in production environments is and should be low as it's still in experimental form. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Lennart Regebro wrote:
On Nov 10, 2007 5:12 AM, Tres Seaver <tseaver@palladion.com> wrote:
If the REST stuff depended on unreleased versions of upstream packages, then it should not have been merged until those packages were released in an acceptable (non-snapshot) form.
That's a good sentiment and I agre with it in practice. But Grok is still in < 1.0 status so I don't eally think it's a big issue. Waiting for the zope packages to stabilize would only have delayed the progress of Grok at this moment, and the amount of people using Grok in production environments is and should be low as it's still in experimental form.
The first point is that whoever put the '.dev-r###' eggs out on a public server made an error: if you don't have the time to do proper release management, then don't release at all. We've spent more effort on the argument now than it would have taken to do the releases. That said, if merging a feature branch requires picking up unreleased packages, then the merge should not be done, period: merging in that state is like merging with failing unit tests. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHNa3B+gerLs4ltQ4RAnqwAJ42iOMN36WusKLCBykn8N5ttT7/vACfUkhj p/7FV6Ve19Crb7S7i/1pWAQ= =eXCq -----END PGP SIGNATURE-----
Tres Seaver wrote:
A KGS needs to have the following properties:
- The "generation zero" of any KGS is an empty set of revisions.
- By default, any revision N of a KGS starts out as a "draft" version which is an empty layer over version N-1. Changes to the draft then shadow any versions in the prior revision.
- Once "finalized" / "published", a given revision of a KGS can *never* be changed.
I sympathize with this thinking, but as far as I understood Jim and Stephan last night, there seems to be the goal to make a KGS for a whole stable release branch, e.g. "Zope3.4", and keep on adding bugfix releases. So there doesn't seem to be a consensus yet on how we should manage KGSs.
- Any KGS can be derived from the published version of another KGS, with additions of new distributions / versions and updates of versions for underlying distributions.
- In conjunction with a "find-links" site which promises never to remove any distribution which has been included in a published revision of a KGS, the current 'version.cfg' (and workalike) files are sufficient to establish a KGS. Without that promise, however, those files can't be considered as defining a KGS.
That makes more sense to me than the locked down index.
Therefore,
- Given that all distribution versions mentioned in a KGS are stored at a trusted / reliable URL, any immutable KGS revision can be trivially transformed into a PyPI package index: each distribution will have exactly one allowed version, which will point to a single URL on a reliable server.
Right, I suppose that can be arranged on top of the version.cfg thing. I find the locked down index approach attractive because it works with pure setuptools, and it largely takes the responsibility out of Joe Middleclass Developer's hands. That is also its greatest weaknesses because doing anything that's not in the KGS requires some situps, for instance managing a find-links location with the extra packages you want to install. The locked down index is also exclusive, two orthogonal KGSs are best merged using the version.cfg approach, it seems.
On Nov 10, 2007 10:02 AM, Philipp von Weitershausen <philipp@weitershausen.de> wrote:
I sympathize with this thinking, but as far as I understood Jim and Stephan last night, there seems to be the goal to make a KGS for a whole stable release branch, e.g. "Zope3.4", and keep on adding bugfix releases.
That I think would be a bad idea. Security fixes I'm OK with but not anything else, because as mentioned that will break applications. And you don't want to have a site that works break just because you ran buildout. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Philipp von Weitershausen wrote:
Tres Seaver wrote:
A KGS needs to have the following properties:
- The "generation zero" of any KGS is an empty set of revisions.
- By default, any revision N of a KGS starts out as a "draft" version which is an empty layer over version N-1. Changes to the draft then shadow any versions in the prior revision.
- Once "finalized" / "published", a given revision of a KGS can *never* be changed.
I sympathize with this thinking, but as far as I understood Jim and Stephan last night, there seems to be the goal to make a KGS for a whole stable release branch, e.g. "Zope3.4", and keep on adding bugfix releases. So there doesn't seem to be a consensus yet on how we should manage KGSs.
Doing only that would be like having a release branch but no tags, i.e., unacceptable. Creating "frozen snapshots" (like CVS/SVN) tags after adding and testing new versions shouldn't be any harder than making a tag is today.
- Any KGS can be derived from the published version of another KGS, with additions of new distributions / versions and updates of versions for underlying distributions.
- In conjunction with a "find-links" site which promises never to remove any distribution which has been included in a published revision of a KGS, the current 'version.cfg' (and workalike) files are sufficient to establish a KGS. Without that promise, however, those files can't be considered as defining a KGS.
That makes more sense to me than the locked down index.
Therefore,
- Given that all distribution versions mentioned in a KGS are stored at a trusted / reliable URL, any immutable KGS revision can be trivially transformed into a PyPI package index: each distribution will have exactly one allowed version, which will point to a single URL on a reliable server.
Right, I suppose that can be arranged on top of the version.cfg thing. I find the locked down index approach attractive because it works with pure setuptools, and it largely takes the responsibility out of Joe Middleclass Developer's hands. That is also its greatest weaknesses because doing anything that's not in the KGS requires some situps, for instance managing a find-links location with the extra packages you want to install. The locked down index is also exclusive, two orthogonal KGSs are best merged using the version.cfg approach, it seems.
If we have software which can generate the pacakge index at a URL for a KGS, then it could certainly generate a 'version.cfg' as well, to support buildout. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHNazm+gerLs4ltQ4RAswyAKDaSgJSIw32gckbd9ctLUX+tgt6GQCbB9ON pYBf589Zdj/0aSihzXfzMvY= =RlBH -----END PGP SIGNATURE-----
On 10 Nov 2007, at 14:06 , Tres Seaver wrote:
If we have software which can generate the pacakge index at a URL for a KGS, then it could certainly generate a 'version.cfg' as well, to support buildout.
We already do. Stephan's zope.release does that already. The file is actually available through http://download.zope.org/zope3.4/ versions.cfg (which I wasn't aware of previously either, I always thought the KGS meant having to use the locked down index).
On Friday 09 November 2007, Tres Seaver wrote:
A KGS needs to have the following properties:
- The "generation zero" of any KGS is an empty set of revisions.
Yes, luckily we are far beyond that point.
- By default, any revision N of a KGS starts out as a "draft" version which is an empty layer over version N-1. Changes to the draft then shadow any versions in the prior revision.
Yes, I have to ask Jim to create a "zope-dev" directory for me that will function as the draft KGS.
- Once "finalized" / "published", a given revision of a KGS can *never* be changed.
I think this can be easily accomplished by released versioned versions.cfg files. Starting with the next release I will create these files; they will have the form "versions-x.x.x.cfg", for example "versions-3.4.0b3.cfg". Overall, I originally did not see the use case for totally fixing the versions. But you and Lennart got me convinced. In order to safely deploy an application, you need to fully fix the versions, just to be very sure. So I think the revised workflow looks like this: 1. During development, you depend on the latest stable KGS, using the index option in buildout. This way you get the latest bug-fix releases, but you are *not* interupted by major feature changes. This would be the same as working with a Zope3 release branch before. Of course, if you are brave, you can also develop against the "zope-dev" KGS, which can be compared to working with Zope3 trunk before. 2. Once you you are ready to deploy your application, you can choose a specific versions-*.cfg or even download the latest KGS configuration in versions.cfg (which is more risky, since you might an uncommon set).
- Any KGS can be derived from the published version of another KGS, with additions of new distributions / versions and updates of versions for underlying distributions.
I think this could be easily accomplished by writing a little bit more software. I think the best syntax would be as follows -- i.e.: grok-controlled-packages.cfg:: [KGS] bases = http://download.zope.org/zope3.4/controlled-packages.cfg I think adding this functionality would be straight forward. I guess with urllib2, the protocol could be almost anything.
- In conjunction with a "find-links" site which promises never to remove any distribution which has been included in a published revision of a KGS, the current 'version.cfg' (and workalike) files are sufficient to establish a KGS. Without that promise, however, those files can't be considered as defining a KGS.
Why do you need find-links in the mix? It must be a policy, that any published release can *never* be revoked.
Therefore,
- Given that all distribution versions mentioned in a KGS are stored at a trusted / reliable URL, any immutable KGS revision can be trivially transformed into a PyPI package index: each distribution will have exactly one allowed version, which will point to a single URL on a reliable server.
Why would I need to create a PyPI index, if I want to fix all versions? It is much easier to reference "http://download.zope.org/zope3.4/versions-3.4.0.cfg" (for example) in your buildout file:: [buildout] extends = http://download.zope.org/zope3.4/versions-3.4.0.cfg Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Friday 09 November 2007, Martijn Faassen wrote:
Roger Ineichen wrote:
Hi Tres
Whoever released those two eggs (the '.dev-r#####' ones) need to release "real" updated packages, and then grok 0.11.1 should be released using them.
DEATH TO FAUX PACKAGES!
As far as I understand, this does not happen if you depend on a KGS, right?
Does the grok release not use the KGS from Stephan?
Not yet. We intend to look at that list at some point. Of course the KGS probably doesn't contain recent enough packages to support the REST changes.
Are you sure? What package versions do you need. I am pretty sure that the 3.4 KGS has the latest stuff, except for the zope.app.publisher 3.5.xa* series, which caused a lot of havoc.
The other problem with KGS is that I absolutely want to fix lists of packages into Grok itself and never rely on any system that can cause the list of packages that could be updated. KGS will get bugfix releases. These will inevitably break something on occassion. I have no idea what will happen once 3.5 packages will start to appear, either.
While I personally do not think that this is a good idea, I am open to the suggestion to fix the KGS versions for a particular Zope 3.4 release. For example, instead of having just a versions.cfg, we can expand the scripts to also produce release versions, such as "versions-3.4.0b2.cfg", which would never change. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Nov 10, 2007 12:52 PM, Stephan Richter <srichter@cosmos.phy.tufts.edu> wrote:
While I personally do not think that this is a good idea, I am open to the suggestion to fix the KGS versions for a particular Zope 3.4 release. For example, instead of having just a versions.cfg, we can expand the scripts to also produce release versions, such as "versions-3.4.0b2.cfg", which would never change.
Hmmm. Could the versions.cfg be checked into the svn? That way we would automatically have both fixed versions on the tags, and versions on the branches... Or maybe this makes no sense in the light of the scripts you are using. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
On Saturday 10 November 2007, Lennart Regebro wrote:
On Nov 10, 2007 12:52 PM, Stephan Richter <srichter@cosmos.phy.tufts.edu> wrote:
While I personally do not think that this is a good idea, I am open to the suggestion to fix the KGS versions for a particular Zope 3.4 release. For example, instead of having just a versions.cfg, we can expand the scripts to also produce release versions, such as "versions-3.4.0b2.cfg", which would never change.
Hmmm. Could the versions.cfg be checked into the svn? That way we would automatically have both fixed versions on the tags, and versions on the branches... Or maybe this makes no sense in the light of the scripts you are using.
Right, I am already correctly branching the zope.release package. It contains the controlled-packages.cfg. Using the scripts in zope.release, it is trivial to render versions.cfg; I mean as in: ./bin/generate-versions I have not created a tag for Zope 3.4.0b2, because I was not thinking of the KGS to function on the specific release level, but on the major release level. However, in light of the recent discussion, I think tagging zope.release is a good thing and I will do this from now on. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On Nov 10, 2007 1:06 PM, Stephan Richter <srichter@cosmos.phy.tufts.edu> wrote:
Right, I am already correctly branching the zope.release package. It contains the controlled-packages.cfg. Using the scripts in zope.release, it is trivial to render versions.cfg; I mean as in: ./bin/generate-versions
I have not created a tag for Zope 3.4.0b2, because I was not thinking of the KGS to function on the specific release level, but on the major release level. However, in light of the recent discussion, I think tagging zope.release is a good thing and I will do this from now on.
Great stuff. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
participants (7)
-
Lennart Regebro -
Martijn Faassen -
Martin Aspeli -
Philipp von Weitershausen -
Roger Ineichen -
Stephan Richter -
Tres Seaver