[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like

[Jim Fulton]

On Jan 25, 2007, at 5:09 PM, Ian Bicking wrote:

> Whit pointed me to this thread.

Yeah, someone pointed me to it too. :)

>   I won't reply to specifics, but maybe
> just describe what we're doing (and planning to do), and how  
> workingenv
> differs from zc.buildout.

I'll avoid responding to general qualitative statements.

...

> I actually tried to do this once before with zc.buildout, but I didn't
> get far -- probably a result of lack of effort and lack of familiarity
> with the overall stack.  But I also recognize lots of the questions
> about stuff like the zope.conf file and Data.fs that still seem
> unresolved.

Certainly when you tried this, buildout was very young and we hadn't  
written recipes to deal with these issues.  We've made a lot of  
progress since then.


> The thing that frustrated me with zc.buildout is that I
> knew how to do what I wanted to do, but I still felt like I was a long
> way from being able to do it.  And little things, like unhelpful error
> messages

Yeah, buildout still needs to do a lot better with error messages,  
although it has probably made some progress since you tried it.

> and frustratingly slow performance really killed my motivation.

That has improved quite a bit.

...

> And frankly I like easy_install.  It's
> probably 10x faster than buildout.

I doubt that that is true now.  Although that probably depends on  
what you are doing.  Early versions of buildout did a lot of things  
inefficiently as I was still learning setuptools.  Because of the way  
that buildout caches index information, I expect that creating a  
buildout from scratch that used a lot of eggs would be much faster  
than using easy_install.  One difference though is that buildout  
checks for the most recent compatible versions of all of the eggs  
it's using every time you run it, whereas, as I understand it, with  
workingenv, you'd just run easy_install manually when you want a new  
egg.  You can bypass the checks by running in offline mode.  Then  
buildout runs very fast.  Because of the ability to share eggs  
accross buildouts, it is often possible to run a buidout using lots  
of eggs in offline mode.

It has been suggested that there should be a mode for buildout that  
only talks to the network when there isn't a local egg that satisfied  
a requirement.  This would make buildout work more like workingenv  
when few if any eggs are actually needed.


> easy_install is what people use in
> documentation, and its conventions are the ones people know (why does
> buildout not use "Pkg==version", for instance?).

It does.  When specifying eggs, you use standard setuptools  
requirement syntax.



> As for the technical reasons they don't work together:
>
> * workingenv allows and leaves it to setuptools to maintain the  
> package
> installation database (basically easy-install.pth).  This is not a  
> very
> good database, but eh.  buildout doesn't really have a database, but
> instead just enforces what buildout.cfg indicates.

buildout uses the buildout configuration file to store what you  
want.  It uses .installed.cfg to capture what you have.  These are  
both databases of sorts.


> * workingenv relies on that database to give default versions and to
> setup the Python path.  The fixup it does of installed scripts is  
> fairly
> minimal, just setting up sys.path enough to force its site.py to get
> called.  buildout enumerates all the activated packages, and ignores
> easy-install.pth.  This is basically what makes it
> easy_install-incompatible.

Yup.  I wanted something far more static and predictable for scripts  
generated by buildout.

> Plus buildout's desire to own everything and
> destroy everything it does not own ;)

I'm not aware that it destroys anything. Could you be more specific?

>
> * As a result buildout supports multiple things in the same buildout
> that have conflicting version requirements, but where the packages
> themselves don't realize this (but the deployer does).  If the  
> packages
> know their requirements then setuptools' native machinery allows  
> things
> to work fine.

Yes.  I expect that usually, packages won't be very specific.  The  
buildout configuration file provides a place to be specific.

> * Some see bin/activate as a jail.  Both workingenv and buildout are
> deliberately jail-like.  Both Jim and I loathe the non- 
> repeatability of
> system-wide installations (at least I think I can speak for him on  
> that
> one point ;).  bin/activate lets you into that jail, and lets you work
> there.  There is no way into a buildout.

I'm not familiar with bin/activate, but it sounds like an interpreter  
script created with buildout.



>   Frankly this weirds me out,
> and is a big part of my past frustration with it.  Maybe that's  
> because
> I'm in the relatively uncommon situation that I actually know what's
> going on under the hood of Python imports and packaging, and so it
> bothers me that I can't debug things directly.  Anyway, neither  
> requires
> activation when using scripts generated in the environment.  And
> bin/activate is really just something that sets PYTHONPATH and then  
> does
> other non-essential things like changing the prompt and $PATH -- I
> should probably document that more clearly.

sounds a lot like an buildout interpreter script.

>   Neither can be entirely
> compatible with a system-wide Python installation, because Python's
> standard site.py f**ks up the environment really early in the process,
> and avoiding that isn't all that easy.

This reminds me of a place where buildout is looser than workenv.   
buildout doesn;t try to disable anything in the system python.  It  
just augments it.  I always use a clean python, so avoiding  
customizations in the Python I use isn't a problem.  If I wanted to  
take advantage of something in a system Python, as I occasionally do,  
I can do that with buildout.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org