[Grok-dev] solving buildout problems - pinning down eggs

[Martijn Faassen]

Hey,

Jasper yesterday (see earlier message) ran into yet another case where 
grokproject wouldn't work because of some new egg release. I've run into 
several problems now over the last weeks, too.

Let me be blunt about this: This is a critical problem for Grok. It's 
pretty atrocious: we can't do a Grok release and then have it break on 
us, several times, in the next few weeks! We might lose countless people 
who just wanted to try out Grok just by problems like this.

How do we solve this immediate problem for grok? What has this 
requirement for zope.traversing that trips us up?

We can't keep solving these individual problems over and over again, as 
it's clear it'll never end. So, on to solving the general problem:

We need to have, and use, the facility to "pin down" Grok's dependencies 
to specific version numbers. This means we need to produce a complete 
list of *all* of the egg versions that grok is using. I want to pin it 
down exactly - all dependencies are liabilities and we can't allow a 
single one to break everything.

We then need to have an easy way to make sure that people who use Grok 
x.y actually use those versions, by default.

Ideally we also want the ability for people to use newer versions of 
eggs, if they so choose. This is a bonus feature in my mind. If this 
bonus feature takes time to work out, we need to forget about it for 
now, as this problem is critical.

Another requirement is that this should be easy for release managers. 
Not a lot of work to produce, and fit with the current egg release 
process ("python setup.py bdist_egg upload"). I think we shouldn't 
introduce a separate release artifact that needs separate management - 
I'd like it to work with eggs.

I know Philipp had a discussion on Zope3-dev on this topic. Philipp, 
what is the outcome of this discussion? Any clear path forward? I got 
the impression the suggestion was to try to use distribution eggs that 
pin version numbers, but we need to work out the details and see whether 
this truly fulfills our requirements.

If making this work takes time, I propose we produce a 0.10.1 release 
which simply pins down all the dependencies in its setup.py, hard. I 
know that breaks the requirement that people should be able to upgrade 
to newer versions of dependencies if they so choose, but we do know it 
actually works. We need to get rid out of this swamp of buildout problems.

Regards,

Martijn