I wanted to try using the snowsprint-viewlets2 branch of grok in my project the other day. It took me a little time to figure out how to do this, so I thought it'd be nice if there was a bit of documentation on how-to pull in a development version of Grok into a grok project, so I wrote this: http://grok.zope.org/documentation/how-to/trail-blazing (I've configured the Grok site to allow OpenId for authentication, and granted the View and Reply to Item permissions to documentation that is in the "Waiting for Review" state, which means you need to log-in to see the document, but that anyone can log-in, which seems like a decent compromise for unreviewed documentation on the Grok site which is potentially incorrect.) The part where I was fuzzy in the documentation process was the terminology for Known Good Sets. There are Grok releases such as "Grok 0.11" which has been called "pinned versions" (or "nailed versions") and when writting my help doc I've been calling this a "Known Good Set". I would propose this definition for Known Good Set: Known Good Set : A frozen set of Python egg names and version numbers that have been tested to work together. This set of eggs is also available from an archive maintained by the publisher of the known good set. For Grok this would be: http://grok.zope.org/releaseinfo/grok-0.11.cfg and the eggs are archived at: http://download.zope.org/distribution/ (at least that's my understanding of the Grok releases, maybe I don't have that quite right ... ) However, the Zope 3.4 release announcement varies a bit from my understanding of Grok-centric understanding of Known Good Set: "The known good set -- or in short KGS -- is a package index that derives from the official Python Package Index (PyPI) and thus contains all available packages in the Python world. But for a controlled set of packages, only certain versions that are known to work together are available. " This is the part that seems a bit confusing to me, since it's not clear if the "set" also includes the super-set of packages from PyPI. It's also not clear if the super-set of all PyPI packages is also a continually updated mirror, or a frozen index. Or is the known good set just the controlled set of packages? It's also not entirely clear on how maintenance releases happen with Zope 3.4 from this, e.g. one could interpret the contents of the versions.cfg URL as either "frozen" (Zope 3.4.0-final) or "latest". For maintenance releases would there be: http://download.zope.org/zope3.4/versions.cfg http://download.zope.org/zope3.4.1/versions.cfg http://download.zope.org/zope3.4.2/versions.cfg Or if this URL will always contain the latest stable packages in the 3.4 series then perhaps this should be more explicit with an URL such as: http://download.zope.org/zope3.4/current-versions.cfg Shouldn't there also be URLs such as these? http://download.zope.org/zope3.4/3.4.0c1-versions.cfg http://download.zope.org/zope3.4/3.4.0-versions.cfg http://download.zope.org/zope3.4/3.4.1-versions.cfg It should also be more explicit what "controlled" means. It seems like this is the same set of packages and versions and versions.cfg except that it also contains versions of previous packages? Is this a continually updated set of the most current versions of all packages that make up the Zope 3.4 series after all test suites pass? http://download.zope.org/zope3.4/controlled-packages.cfg Too me it seems easier to understand if there are two URLs in buildout's find-links setting. e.g. for Plone there is a link to a controlled archive of Plone produced packages, then there is a link to PyPI (well the PPIX mirror actually), which makes it more obvious that there is a Zope managed archive of packages, and then there is the wide, wooly world of all Python packages: find-links = http://dist.plone.org http://download.zope.org/ppix/ Anyways, I hope I don't sound too complain-y, but it would be much appreciated if the terminology and plan for maintaining the Zope 3.4 release series was made a bit clearer. And it would also be really nice if the Grok terminology and release methods lined up with the Zope 3 terminology and release methods :)
On Feb 3, 2008, at 3:35 AM, Kevin Teague wrote:
I wanted to try using the snowsprint-viewlets2 branch of grok in my project the other day. It took me a little time to figure out how to do this, so I thought it'd be nice if there was a bit of documentation on how-to pull in a development version of Grok into a grok project, so I wrote this:
I couldn't access this page. The easiest way to use grok trunk/branch is to use svn:externals and change a few lines in buildout.cfg [buildout] develop = . src/grok src/grok/martian [versions] martian = and put this in svn:externals for src/ grok svn://svn.zope.org/repos/main/grok/branches/snowsprint-viewlets2
Anyways, I hope I don't sound too complain-y, but it would be much appreciated if the terminology and plan for maintaining the Zope 3.4 release series was made a bit clearer. And it would also be really nice if the Grok terminology and release methods lined up with the Zope 3 terminology and release methods :)
Grok should use KGS. Someone just needs to do the work. /Tim
On Feb 3, 2008, at 10:50 AM, Tim Terlegård wrote:
On Feb 3, 2008, at 3:35 AM, Kevin Teague wrote:
I wanted to try using the snowsprint-viewlets2 branch of grok in my project the other day. It took me a little time to figure out how to do this, so I thought it'd be nice if there was a bit of documentation on how-to pull in a development version of Grok into a grok project, so I wrote this:
I couldn't access this page.
You should be able to see the page if you log-in? It's not published because I wanted to get the terminology straight first. I may also re-work the text a bit. At first I was just going to do a very short "How-to develop from a trunk/branch" thing and I marked the Audience field as "Advanced Developer" but then I ended up writting a bit more about eggs and buildout in general, since I think we could use a better explanation of *why* eggs and buildout were developed (and why they are so cool). Otherwise when comparing Zope 3/Grok with other frameworks at first glance it seems like there is just a whole bunch of extra complexity and you don't really understand the benefits until you've spend a fair bit of time learning and working with the tool. I did get Viewlets working in my Grok project, good stuff :)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kevin Teague wrote:
I wanted to try using the snowsprint-viewlets2 branch of grok in my project the other day. It took me a little time to figure out how to do this, so I thought it'd be nice if there was a bit of documentation on how-to pull in a development version of Grok into a grok project, so I wrote this:
http://grok.zope.org/documentation/how-to/trail-blazing
(I've configured the Grok site to allow OpenId for authentication, and granted the View and Reply to Item permissions to documentation that is in the "Waiting for Review" state, which means you need to log-in to see the document, but that anyone can log-in, which seems like a decent compromise for unreviewed documentation on the Grok site which is potentially incorrect.)
The part where I was fuzzy in the documentation process was the terminology for Known Good Sets. There are Grok releases such as "Grok 0.11" which has been called "pinned versions" (or "nailed versions") and when writting my help doc I've been calling this a "Known Good Set". I would propose this definition for Known Good Set:
Known Good Set : A frozen set of Python egg names and version numbers that have been tested to work together. This set of eggs is also available from an archive maintained by the publisher of the known good set.
For Grok this would be:
http://grok.zope.org/releaseinfo/grok-0.11.cfg
and the eggs are archived at:
http://download.zope.org/distribution/
(at least that's my understanding of the Grok releases, maybe I don't have that quite right ... )
You have one of the tow (disputed) definitions right (and I agree with you).
However, the Zope 3.4 release announcement varies a bit from my understanding of Grok-centric understanding of Known Good Set:
"The known good set -- or in short KGS -- is a package index that derives from the official Python Package Index (PyPI) and thus contains all available packages in the Python world. But for a controlled set of packages, only certain versions that are known to work together are available. "
This is the part that seems a bit confusing to me, since it's not clear if the "set" also includes the super-set of packages from PyPI. It's also not clear if the super-set of all PyPI packages is also a continually updated mirror, or a frozen index. Or is the known good set just the controlled set of packages?
Agreed: the'package index he refers to is a mirror of PyPI, with the exception that packages tracked by the KGS have those versions, rather than whatever is in PyPI. The rationale (which I disagree with) is that people want to use an index, but aren't willing to switch indexes when installing a package not tracked by the KGS. I have argued fairly strongly in the past that this is a bad idea, since there is no possiblity that the indexed packages are "known good".
It's also not entirely clear on how maintenance releases happen with Zope 3.4 from this, e.g. one could interpret the contents of the versions.cfg URL as either "frozen" (Zope 3.4.0-final) or "latest". For maintenance releases would there be:
I would argue that this page shouldn't exist yet, because 3.4 final has not yet been released. It shod be something like: http://download.zope.org/zope3.4.0c1/versions.cfg
http://download.zope.org/zope3.4.1/versions.cfg http://download.zope.org/zope3.4.2/versions.cfg
Those pages don't exist yet, because there are no releases later than 3.4.0.
Or if this URL will always contain the latest stable packages in the 3.4 series then perhaps this should be more explicit with an URL such as:
+1: if they represent a set which is changing over time, then they should be named in a way that represents that: otherwise, people may expect repeatable results when using the page.
Shouldn't there also be URLs such as these?
http://download.zope.org/zope3.4/3.4.0c1-versions.cfg http://download.zope.org/zope3.4/3.4.0-versions.cfg http://download.zope.org/zope3.4/3.4.1-versions.cfg
Something like that. ;)
It should also be more explicit what "controlled" means. It seems like this is the same set of packages and versions and versions.cfg except that it also contains versions of previous packages? Is this a continually updated set of the most current versions of all packages that make up the Zope 3.4 series after all test suites pass?
I don't nnow what that page means, exaclty.
Too me it seems easier to understand if there are two URLs in buildout's find-links setting. e.g. for Plone there is a link to a controlled archive of Plone produced packages, then there is a link to PyPI (well the PPIX mirror actually), which makes it more obvious that there is a Zope managed archive of packages, and then there is the wide, wooly world of all Python packages:
find-links = http://dist.plone.org http://download.zope.org/ppix/
Anyways, I hope I don't sound too complain-y, but it would be much appreciated if the terminology and plan for maintaining the Zope 3.4 release series was made a bit clearer. And it would also be really nice if the Grok terminology and release methods lined up with the Zope 3 terminology and release methods :)
+1. I think we are going to find that projects need to maintain their own private indexes, just for the sake of sanity / repeatability. I've done a litte work on building package indexes based on the packages which are actually available to 'pkg_resources': http://svn.repoze.org/compoze/trunk/README.txt http://dist.repoze.org/compoze-0.2.tar.gz I think that will work from a buildout-driven setup, as long as you hack PYTHONPATH the 'compoze' script in the same way buildout hacks all the other scripts (I normally use a virtualenv, which means I don't need to worry about munging the path in each script). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHphTF+gerLs4ltQ4RAq3GAJ4qVR57Lq5ZJSite7gvrvIOOi2w2ACggHav 6rlYi4vJDGqKQ2lY+o2Ht1Q= =UG8P -----END PGP SIGNATURE-----
Hey, Kevin Teague wrote:
And it would also be really nice if the Grok terminology and release methods lined up with the Zope 3 terminology and release methods :)
Grok got there first, then KGS came along and did a lot of stuff. The reason Grok's story is not the same as the one in KGS is that we haven't caught up with the new stuff yet. We of course reserve the right to simplify it for our purposes. :) What seems most useful about KGS is that it tests versions of eggs together and publishes a list of those versions that are known to work together. The Grok process is fairly simple: * when we release grok, we also release a versions.cfg on grok.zope.org/releaseinfo. We name it after the new release of Grok, and don't ever change it anymore. * we also update a special URL that grokproject makes use of to determine the last release of Grok. This way a newly created grokproject automatically gets configured to use the right releaseinfo URL * grokproject then pins those versions in buildout.cfg using the 'versions = versions' and 'extends = http://grok.zope.org/releaseinfo/some-release-versions.cfg' lines in [buildout]. We don't have any special index servers or anything. Right now we point to PyPI and download.zope.org/distribution; the latter will likely eventually go away as all eggs are properly on the cheeseshop. We do have some issues concerning development, which I think you ran into by looking at the viewlet branch. We can probably simplify the branch by now as we did release a new version of Martian that has the required changes. We intend to try to go with KGS's list of packages. We need to test that and determine the best way to reuse that list. I don't think we'll do much else for now - Grok's process works fairly well for us. One thing that we should eventually improve is to allow grokproject and Grok buildouts to work offline. Right now buildout.cfg refers to a URL in the 'extends' line which breaks things. Another thing I'd like us to experiment with is to offer a one-tarball download of Grok, possibly in combination with a frozen repository which contains all Grok dependencies for a particular release. On the longer term I am still curious about experimenting with getting the "these versions of my dependencies are tested to work" information into the eggs themselves somehow. Regards, Martijn
participants (4)
-
Kevin Teague -
Martijn Faassen -
Tim Terlegård -
Tres Seaver