[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like

[Jim Fulton]

I hate to jump into this thread .... but I'll make a few comments.

On Jan 25, 2007, at 5:13 PM, whit wrote:

> Philipp von Weitershausen wrote:
>> whit wrote:
>>> Martin Aspeli wrote:
>>>> Philipp von Weitershausen wrote:
>>>>
>>>>> This is awesome, and by that I don't mean the fact that we have  
>>>>> a plone buildout, but that we actually have Zope 2 recipes for  
>>>>> buildout. I hope they can be moved to svn.zope.org for further  
>>>>> development to benefit the whole Zope 2 community.
>>>>
>>>> I believe this is just a matter of contrib agreements being  
>>>> sorted out (Hanno?). I guess I need to get mine sorted out as  
>>>> well if I'm going to keep working on this when it moves... :-/
>>>>
>>>>> I also see that workingenv was abandoned. That's very good to  
>>>>> hear because buildout has a lot of machinery for installing  
>>>>> eggs already, it would just've been duplicated with workingenv...
>>>
>>> is there some advantage to the way that buildout handles eggs  
>>> over workingenv. as I understand it, workingenv *only* handles  
>>> python setup and does that well and transparently.

Right, buildout handles a lot more.  For example, we use buildout to  
build C libraries and to generate various sorts of application  
artifacts like configuration files,  custom startup scripts, and so  
on, in addition to Python scripts.


>> The point is that buildout *already* handles eggs. There's really  
>> no point for having an extra layer on top of buildout. The  
>> zc.recipe.egg recipe can install any egg (as a development one or  
>> not) in an automated fashion, which is exactly what you'd want  
>> from a buildout.
>>
> well that's awesome that buildout has duplicated this effort....

I'm not sure what effort you think buildout is duplicating.  WRT  
eggs, buildout, like workinenv/easy_install builds on setuptools.   
setuptools does the heavy lifting with regard to building eggs.   
Buildout does have some nice features for building eggs with  
extensions that need custom setup options.

Like workenv, buildout supports installing eggs in development  
environments rather than the system Python.  workingenv bends  
easy_install to it's will, while buildout goes below easy_install.   
This allows buildout to have more control, which was important to  
me.  Buildout gives me greater control over egg revisions used and  
the way that scripts get generated. (Maybe workingenv would give me  
more control than I think it does -- I haven't looked at it in a while.)


>>> the "source bin/activate" dance is the only thing I see being a  
>>> detriment here(and with the latest workingenv, your shell prompt  
>>> lets
>>> you know you are in an env).
>>
>> Not everybody likes the activate dance. With buildout, you don't  
>> need it. The recipes make sure that the scripts that get installed  
>> into the buildout's 'bin' directory have the right PYTHONPATH set  
>> and have access to the eggs you requested for the buildout.
> is there really a difference between zopectl and source bin/ 
> activate?  I guess the main difference here is where PYTHONPATH  
> gets set and how people work with it.   for example, if you just  
> start python and want to play with code.... sounds like with  
> zc.buildout you are out of luck for things installed in the buildout.

I'm not sure what you mean here. I'll note that buildout lets you  
easily create "interpreter" scripts.  When run without arguments, you  
get a Python prompt.  When run with a script name and arguments, then  
they will run the script with the arguments.  You define an  
interpreter by specifying on or more eggs and those eggs and their  
dependencies (and additional paths you can optionally set) will be  
included in the interpreter's path.  This makes playing with code  
pretty easy.


> In our situation, we might have multiple versions of software  
> running on different processes started in different workingenv  
> (often not even using zope).    being able to contextualize the  
> python path is a useful development tool; this is understandable  
> for buildout to avoid(as a deployment tool), but if we are  
> considering using buildout as a prescribed way for people to manage  
> their development environments, it seems lacking.

I don't know what you mean by "contextualize the python path".  An  
important feature of buildout (actualy the egg recipe) is that it  
lets you create scripts with exactly the eggs they need.  This was a  
core motivation of buildout.  And, as I have mentioned above, you can  
also create interpreter scripts for experimentation and running other  
scripts. Perhaps these interpreter scripts are like your workingenvs.


>>>> Workingenv made it more complex than it needed to be (or  
>>>> buildout did, depending on which perspective you look at it  
>>>> from). I believe Hanno wanted to rescue the recipe in case  
>>>> others found it useful, but it's not used for now.
>>>
>>> what about if I'm already using workingenv... and want to use  
>>> zope or plone in my workingenv?
>>
>> I see no problem with that. zc.buildout is one way of deploying  
>> Python software like Zope. As long as you've got eggs, you could  
>> install them manually into your workingenv just fine. buildout  
>> just does it an automated fashion (and, of course, it can do more  
>> than just installing eggs).
>>
> which I could automate in my env (workingenv takes recipes also).  
> for that matter, eggs automate the installation of eggs...

Actually, they don't. They allow automation and workingenv and  
buildout provide automation, both building on setuptools.


> I'm not sure what it would take to make buildout just automate  
> install eggs into my  env...

I'm not clear on what the advantage would be.  I'm probably missing  
some use cases.  I think they are both valid approaches to the problem.


> but not putting them in special directories would be a start.

Um, both workingenv and buildout install eggs into specific/special  
directories.
You can configure buildout to install eggs wherever you want,  
including into the directory structure that workingenv uses.

Note that buildout separates regular and develop eggs to allow  
regular eggs to be shared across projects. This can be a big win for  
performance, space and allowing buildouts to be created offline.


>
>>> currently, I don't see an easy way to use buildouts inside a  
>>> workingenv, whereas the rest of python world works great.  I will  
>>> have alot of trouble explaining to my developer who already think  
>>> zope smells that they have to change the way they work to use  
>>> zc.buildout recipes.
>>>
>>> for example, I can't use the deliverance or lxml buildout with an  
>>> existing topp.deploy workingenv because of buildout's arbitrary  
>>> egg handling scheme.  If zc.buildout didn't try to do so much,  
>>> the python would be installed transparently like everything else  
>>> I easy_install.
>>
>> I'm not too fond of zc.buildout's directory scheme eiher. In  
>> particular, I wish that it would use 'lib/python' instead of  
>> 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough  
>> zc.buildout details to be able to say whether that can be chagned  
>> by remimplementing the egg installation recipe. I would sure hope  
>> it is.

You can set the eggs directory to anything you want:

[buildout]
eggs-directory = lib/python
develop-eggs-directory = lib/python

If you want, you could put this in ~/.buildout/default.cfg and  
buildout will work this way for all of your buildouts.


> this may be all that is required to make the two play well together 
> (or have buildout respect an existing workingenv when doing it's  
> local install of eggs).  Maybe it's just a matter of zc.buildout  
> seeing workingenv is in effect and not composing special pythonpaths.
>

> harmony is my interest here. workingenv is pretty low-level and  
> works hard to work well with python. there is no reason that  
> zc.buildout should have a problem with that.. it just needs to do  
> less when appropriate.

If people really see value in having buildout and workingenv work  
together, perhaps Ian and I could explore this a bit at PyCon next  
month.

> try to justify why a zope hammer is better and why zope folk keep  
> building new hammers that don't play nice with everyone else's  
> hammer loses us cred.

You seem to be implying that workingenv is somehow a standard tool  
and that buildout is a zope-specific tool that needlessly duplicates  
a standard tool.  zc.buildout is in no way zope specific.  Can a Zope  
developer not develop a tool without it being stamped as "zope  
specific"?  zc.buildout and workingenv started life at about the same  
time.  They are both valid and useful tools with somewhat different  
goals and approaches.  They both deserve consideration.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org