[Grok-dev] a grok installer download Was: Re: remove the download tarball functionality from grokproject?

[Martijn Faassen]

Hi there,

Martijn Faassen wrote:
> Can we make progress on such an installer?

I experimented with this myself for a little while. I haven't gotten 
very far, except to come up with more requirements and ideas, which I'll 
share here.

A description of the installer:

* the installer is downloaded from grok.zope.org as a tarball, like 
grokinstaller-1.1.tgz

* the user unpacks the installer, and runs an install script. This script:

   * installs all required eggs into a shared eggs directory (more about 
this later)

   * installs a grokproject script that users can use to create grok 
projects

* when the user uses 'grokproject', it will create grok projects with 
the same layout as it does today, except eggbasket is not in the 
buildout.cfg.

Some more requirements:

* the installer must be self-contained. no network access should be 
needed to install Grok.

   * no network access to download tarballs and binary eggs

   * no network access to download versions lists.

* translation: the installer should contain all required tarballs, eggs
   and versions lists.

* no compiler should be needed to install on Windows

* the buildout.cfg created by grokproject should work on other machines. 
It shouldn't have hardcoded paths in it to, say, eggs directories.

Now some thoughts about two approaches to sharing eggs.

The approach we have now is the following:

* we recommend virtualenv --no-site-packages to everybody

* we install grokproject into this, along with its dependencies

* grokproject ensures there is a shared eggs directory

* it does this by creating a default.cfg buildout for you

We could follow this approach, where the installer does all of the above 
(create virtualenv, installs grokproject into it, create shared eggs dir 
and default buildout.cfg)

One drawback of the default.cfg is that it is global. It's essential if 
we don't want people to download their eggs over and over again, but 
it's also interfering with any and all buildouts. Normally that's all 
right, but I've had a complaint from Chris McDonough a while back that 
it screwed up repoze.bfg installs.

Our use of virtualenv is sort of half-hearted. We do it because not 
using it leads to too many installation problems. There is work being 
done on a buildout version that can isolate itself properly (like 
--no-site-packages), and possibly other features, but it isn't there 
yet. We should investigate what features this buildout offers, however, 
as it might fix our issues.

Another approach would be to use virtualenv *more*. Instead of 
installing a shared eggs cache, we download all of grok's requirements 
into the site-packages of the virtualenv that our installer creates. 
This will then be our shared eggs cache. It does away with the need of a 
centralized shared eggs cache installed and the need for a centralize 
default default.cfg for buildout. One consequence would be that any 
*extensions* that the user downloads into their buildout would not be 
shared but in the local 'eggs' of each buildout. Is that good or bad?

Another consequence would be that we'd need a way to guarantee that the 
right versions are installed into site-packages, as we can't use 
buildout to do it (or could we hack it to?). Pip?

Given these consequences, I am still inclined to stick with the shared 
egg cache approach, even though it does force us to install a 
default.cfg centrally. I think end-users end up with a nicer setup as a 
result.

It'd be nice if we could somehow avoid this, but I can't see a present 
or future buildout feature right now that would help there (besides a 
long commandline option to bin/buildout, or an alternative bin/rebuild 
script that uses this commandline option. Neither are attractive).

Thoughts?

Regards,

Martijn

P.S. For future reference I've attached my experimental grokinstaller 
buildout.cfg. To create project 'foo', use "bin/grokproject 
--eggs-dir=/wherever/i/unpacked/grokinstaller/eggs foo". But this will 
create a hardcoded 'eggs-directory' in the generated buildout.cfg, which 
is wrong. We need to do the default.cfg dance instead.

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: buildout.cfg
Url: http://mail.zope.org/pipermail/grok-dev/attachments/20100302/0aed5382/attachment.pl