[Grok-dev] grokproject/buildout problems

[Martijn Faassen]

Hi there,

There are a bunch of problems with grokproject/grok buildout that I'd 
like to see investigated and addressed. The big, big problem we need to 
fix sooner rather than later is that the grokproject generated buildouts 
are actually fundamentally broken - I can't run bootstrap.py for a 
grokproject generated project in a fresh environment and have things 
work! We can't live with that...

The problems:

enter a fresh virtual environment
remove .buildout and everything in it, etc
$ easy_install grokproject (0.8)
$ grokproject something

* "Download error: unknown url type: svn -- Some packages may not be 
found!". non-urgent but very long standing and confuses people

* "Couldn't find index page for 'grok' (maybe misspelled?)" - what index 
page? What is triggering this?

* "warning: no files found matching '0.1.0-changes.txt'"

* "no previously-included directories found matching 'docs-in-progress'"

* "eggbasket: Failed to install required eggs with the tar ball. 
Continuing with buildout instead." - repeating twice

The generated project does work in the sense that bin/zopectl is there 
starts.

Now create a new fresh virtual environment, activate it, and go into 
'something', removing this:

$ rm -rf develop-eggs/
$ rm -rf .installed.cfg
$ rm -rf parts/
$ rm -rf bin/

and also putting aside .buildout again:

after this, do:

$ python bootstrap.py

* One thing I notice is a lot of duplication of messages:

Getting distribution for 'z3c.recipe.eggbasket==0.3.1'.
Getting distribution for 'z3c.recipe.eggbasket==0.3.1'.
Got z3c.recipe.eggbasket 0.3.1.
Got z3c.recipe.eggbasket 0.3.1.

basically all messages are duplicated (and these are a lot of 
messages!). Why? Needs to be fixed

* "eggbasket: Failed to install required eggs with the tar ball. 
Continuing with buildout instead." (duplicated). After this "continuing 
with buildout instead" doesn't do anything - it just stops here. We 
don't end up with a working buildout!

* Because I happen to know what's up with buildout, I run 
'bin/buildout'. It seems to be getting the docutils 0.4 distribution 
online for some reason - why isn't this in the eggbasket tarball? Or 
perhaps it is and it isn't picked up for some reason? The whole point of 
it after all is to avoid such dangerous side-trips to sourceforge..

* Things do appear to work then.

Another issue I'm noticing now:

* I installed grokproject already in one virtual environment. Now in 
another virtual environment I install grokproject again and run it. It 
now downloads something big (it doesn't seem to specify what, but it 
must be eggbasket). Then I get:

* "eggbasket: Failed to install required eggs with the tar ball. 
Continuing with buildout instead." Does it not pick up the stuff in 
.buildout/eggs? Do we need a separate one per virtual env? If we use 
virtualenv anyway (it looks more and more inevitable and we already 
recommend it in various places), can we do away with our egg cache there 
altogether and simply use the virtual env's site-packages?

* Even after this it takes forever, still not outputting what it is 
doing. I happen to be on a slow network right now so I probably notice 
these problems more quickly. When I run grokproject twice while in the 
same virtualenv, starting without .buildout directory, grokproject 
*does* run quickly the same time. I wonder what causes this...

Regards,

Martijn