[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like

[Ian Bicking]

Martin Aspeli wrote:
> 
>>>  I don't have a usecase for executing the scripts with any python
>>> interpeter other than the one which ran setuptools to generate them, and
>>> therefore don't care for the hard-wired path manipulation
>>
>> I would agree that having to mangle multiple scripts is annoying. On 
>> the other hand, I find the activate/deactivate dance annoying. I 
>> rather think that the scripts that buildout has created should not 
>> require any special activation or introduction to a particular 
>> environment.
>>
>> Perhaps a solution is to let buildout put things in standard places 
>> (as I'm typing this, we now have all the ploneout .egg and .egg-link 
>> files/directories in ${buildout}/lib/python) and then have an optional 
>> workingenv at the root of the buildout that could be activated when 
>> you want that, but let the buildout-managed scripts (e.g. zope 
>> start/stop and test runner) not require this?
>>
>> I have a feeling there's more to this than agreeing on directory 
>> layout though. But I hope it oughtn't be *that much more*.
> 
> I did some experimentation with this. Basically, I did:
> 
>  - make egg-directory and develop-egg-directory in buildout.cfg be 
> lib/python2.4 (which is what workingenv uses)
> 
>  - run the buildout again
> 
>  - create standard workingenv (python workingenv.py .) in the root 
> buildout directory
> 
> The results are fairly encouraging:
> 
>  - buildout-installed eggs don't work in the workingenv jail...
>  - ...but they do if I manually add them to lib/python2.4/easy_install.pth
>  - easy_installed packages (i.e. installed with workingenv's 
> easy_install) are available to zope
> 
> This was only a brief test, but seems to me that if buildout was 
> managing the easy_install.pth file we wouldn't be a million miles off 
> something where the source bin/activate dance was optional and would 
> work as a generic solution, but would not be required for zope's startup 
> and other buildout-managed scripts.

I would assume that buildout is specifically disabling easy_install's 
updating of easy-install.pth -- buildout is still installing packages 
with easy_install, but with some monkeypatches of the command. 
workingenv also monkeypatches easy_install, but only to change script 
creation (as noted in another email) and to keep it from rejecting 
workingenv's site.py.

Anyway, if easy_install was allowed to do its thing then they'd be more 
compatible.

> However, I'm not sure whether and how buildout should manage 
> easy_install.pth in a non-hacky way. A hacky way would be to have a 
> buildout recipie that, if included, would just patch that file up based 
> on buildout['buildout']['eggs'] (i.e. the list of eggs listed in 
> buildout.cfg under the main [buildout] section).
> 
> I don't really understand the purpose of easy_install.pth and why it 
> matters here, though, so suggestions welcome. :)

Eggs exist to pkg_resources (the runtime portion of setuptools) simply 
by being available on a path.  E.g., if you have lib/python/ on the 
path, and lib/python/Foo-1.0.egg/ exists, then if you do 
pkg_resources.require('Foo') that will add lib/python/Foo-1.0.egg/ to 
the path, and you can import from it.  When you can actually import from 
the egg it is "active".  You can also see its entry points at that time, 
but not if it is just available.

If lib/python/Foo-1.0.egg/ is on the path to start with you can import 
from it directly.  The eggs listed in easy-install.pth are put on the 
path, and hence are activated by default.  You can still require a 
different egg version (e.g., Foo==1.1), and I believe setuptools will 
(if it can) switch the path to remove the default activated egg and put 
on the new egg.

If you install a package with easy_install -m ("multiversion") then none 
of the versions go on the path, and eggs will only be added when they 
are specifically required.  Any egg listed in "install_requires" in 
setup.py will get added as soon as you require the original egg, and 
when you run a script from an egg it'll activate that egg as well, so if 
you are thorough with your requirements then none of the eggs need to be 
activated.

One problem if you are using entry points is that if your egg isn't 
activated and you are searching entry points by name (not by egg) then 
your egg has to be activated to work (Buffet and paster create templates 
are both searched by name, for example).  I have expected this to cause 
more problems than it has, but since people have stopped leaning too 
heavily on setuptools versioning (thanks to virtual-python, workingenv, 
and buildout) I think it hasn't come up much.  Anyway, if Zope was to 
start using entry points to replace some of its ZCML (which I personally 
think would be a good idea) this is something to be aware of.

   Ian