[Zope-dev] Re: pinning, nailing and kgs'ing

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kevin Teague wrote:
> I wanted to try using the snowsprint-viewlets2 branch of grok in my  
> project the other day. It took me a little time to figure out how to  
> do this, so I thought it'd be nice if there was a bit of documentation  
> on how-to pull in a development version of Grok into a grok project,  
> so I wrote this:
> 
> http://grok.zope.org/documentation/how-to/trail-blazing
> 
> (I've configured the Grok site to allow OpenId for authentication, and  
> granted the View and Reply to Item permissions to documentation that  
> is in the "Waiting for Review" state, which means you need to log-in  
> to see the document, but that anyone can log-in, which seems like a  
> decent compromise for unreviewed documentation on the Grok site which  
> is potentially incorrect.)
> 
> The part where I was fuzzy in the documentation process was the  
> terminology for Known Good Sets. There are Grok releases such as "Grok  
> 0.11" which has been called "pinned versions" (or "nailed versions")  
> and when writting my help doc I've been calling this a "Known Good  
> Set". I would propose this definition for Known Good Set:
> 
>    Known Good Set : A frozen set of Python egg names and version  
> numbers that have been tested to work together. This set of eggs is  
> also available from an archive maintained by the publisher of the  
> known good set.
> 
> For Grok this would be:
> 
> http://grok.zope.org/releaseinfo/grok-0.11.cfg
> 
> and the eggs are archived at:
> 
> http://download.zope.org/distribution/
> 
> (at least that's my understanding of the Grok releases, maybe I don't  
> have that quite right ... )

You have one of the tow (disputed) definitions right (and I agree with you).

> However, the Zope 3.4 release announcement varies a bit from my  
> understanding of Grok-centric understanding of Known Good Set:
> 
> "The known good set -- or in short KGS -- is a package index that  
> derives from the official Python Package Index (PyPI) and thus  
> contains all available packages in the Python world. But for a  
> controlled set of packages, only certain versions that are known to  
> work together are available. "
> 
> This is the part that seems a bit confusing to me, since it's not  
> clear if the "set" also includes the super-set of packages from PyPI.  
> It's also not clear if the super-set of all PyPI packages is also a  
> continually updated mirror, or a frozen index. Or is the known good  
> set just the controlled set of packages?

Agreed:  the'package index he refers to is a mirror of PyPI, with the
exception that packages tracked by the KGS have those versions, rather
than whatever is in PyPI.  The rationale (which I disagree with) is that
people want to use an index, but aren't willing to switch indexes when
installing a package not tracked by the KGS.

I have argued fairly strongly in the past that this is a bad idea, since
there is no possiblity that the indexed packages are "known good".

> It's also not entirely clear  
> on how maintenance releases happen with Zope 3.4 from this, e.g. one  
> could interpret the contents of the versions.cfg URL as either  
> "frozen" (Zope 3.4.0-final) or "latest". For maintenance releases  
> would there be:
>
> http://download.zope.org/zope3.4/versions.cfg

I would argue that this page shouldn't exist yet, because 3.4 final has
not yet been released.  It shod be something like:

 http://download.zope.org/zope3.4.0c1/versions.cfg

> http://download.zope.org/zope3.4.1/versions.cfg
> http://download.zope.org/zope3.4.2/versions.cfg

Those pages don't exist yet, because there are no releases later than 3.4.0.

> Or if this URL will always contain the latest stable packages in the  
> 3.4 series then perhaps this should be more explicit with an URL such  
> as:
> 
> http://download.zope.org/zope3.4/current-versions.cfg

+1:  if they represent a set which is changing over time, then they
should be named in a way that represents that:  otherwise, people may
expect repeatable results when using the page.

> Shouldn't there also be URLs such as these?
> 
> http://download.zope.org/zope3.4/3.4.0c1-versions.cfg
> http://download.zope.org/zope3.4/3.4.0-versions.cfg
> http://download.zope.org/zope3.4/3.4.1-versions.cfg

Something like that. ;)

> It should also be more explicit what "controlled" means. It seems like  
> this is the same set of packages and versions and versions.cfg except  
> that it also contains versions of previous packages? Is this a  
> continually updated set of the most current versions of all packages  
> that make up the Zope 3.4 series after all test suites pass?
> 
> http://download.zope.org/zope3.4/controlled-packages.cfg

I don't nnow what that page means, exaclty.

> Too me it seems easier to understand if there are two URLs in  
> buildout's find-links setting. e.g. for Plone there is a link to a  
> controlled archive of Plone produced packages, then there is a link to  
> PyPI (well the PPIX mirror actually), which makes it more obvious that  
> there is a Zope managed archive of packages, and then there is the  
> wide, wooly world of all Python packages:
> 
> find-links =
>      http://dist.plone.org
>      http://download.zope.org/ppix/
> 
> Anyways, I hope I don't sound too complain-y, but it would be much  
> appreciated if the terminology and plan for maintaining the Zope 3.4  
> release series was made a bit clearer. And it would also be really  
> nice if the Grok terminology and release methods lined up with the  
> Zope 3 terminology and release methods :)

+1.    I think we are going to find that projects need to maintain their
own private indexes, just for the sake of sanity / repeatability.  I've
done a litte work on building package indexes based on the packages
which are actually available to 'pkg_resources':

  http://svn.repoze.org/compoze/trunk/README.txt

  http://dist.repoze.org/compoze-0.2.tar.gz

I think that will work from a buildout-driven setup, as long as you hack
PYTHONPATH the 'compoze' script in the same way buildout hacks all the
other scripts (I normally use a virtualenv, which means I don't need to
worry about munging the path in each script).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHphTF+gerLs4ltQ4RAq3GAJ4qVR57Lq5ZJSite7gvrvIOOi2w2ACggHav
6rlYi4vJDGqKQ2lY+o2Ht1Q=
=UG8P
-----END PGP SIGNATURE-----