Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote:
I've been playing with this a bit over the past couple of days, and it's now in a state where I understand what's going on and I feel that I may well try to use it in a real project. It still needs a bit of work (notably, the testrunner is struggling to run tests for things in the Products/ directory) and lots more testing, but if you're at all interested in this, now would be a great time to get involved.
I've put up some documentation here:
http://dev.plone.org/plone/browser/ploneout/trunk/README.txt
To use it, you should just have to do this:
$ svn co https://svn.plone.org/svn/plone/ploneout/trunk ploneout $ cd ploneout $ python boostrap.py $ ./bin/buildout -v
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community. I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
Depending on how svn.zope.org is feeling, that may take a while and it uses a lot of space (since it does a full Zope 2 checkout). I would like to explore options for using a different/common Zope 2 checkout and for using a tarball instead of svn to get Zope 2.10 in the future.
In any case, once that's all done, you can do:
$ cd parts/instance $ ./bin/zopectl fg
gocept's Zope 3 instance recipe actually installs a script into the top-level bin directory, so you can do: $ bin/buildout ... $ bin/instance fg Where 'instance' is the name of the instance configuration section in buildout.cfg. So, if you have multiple instances, you can start them all with according scripts from 'bin'. It'd be really cool if z2c.recipe.zope2instance would do the same. To be frank, I think having to mess around in 'parts' sucks. I would even argue that app data like logs, databases, etc. shouldn't be in 'parts' either. You should be able to delete 'parts' and lose nothing (disclaimer: this is my view, not sure how "Buildout Jim" see it ;)).
This is a bonafide Zope 2 instance, but note that the products directory is actually ploneout/products ($INSTANCE_HOME/Products is not used), Data.fs will be placed in ploneout/parts/filestorage (so that buildout doesn't delete it when it wipes the instance), and various eggs are in play found inside ploneout/eggs (those downloaded automatically by buildout) and ploneout/develop-eggs (those originally found in ploneout/src).
Thanks to Hanno for doing 99% of the work on all this. :)
Yeah! We all owe beers to Hanno. -- http://worldcookery.com -- Professional Zope documentation and training 2nd edition of Web Component Development with Zope 3 is now shipping!
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
gocept's Zope 3 instance recipe actually installs a script into the top-level bin directory, so you can do:
$ bin/buildout ... $ bin/instance fg
Where 'instance' is the name of the instance configuration section in buildout.cfg. So, if you have multiple instances, you can start them all with according scripts from 'bin'.
It'd be really cool if z2c.recipe.zope2instance would do the same. To be frank, I think having to mess around in 'parts' sucks. I would even argue that app data like logs, databases, etc. shouldn't be in 'parts' either. You should be able to delete 'parts' and lose nothing (disclaimer: this is my view, not sure how "Buildout Jim" see it ;)).
I would tend to agree, especially since buildout has a tendency to delete things in parts/ in any case (or rather, recipes do). I don't think it'd be hard to make such a script (perhaps more difficult to make it be cross-platform, we really need a champion for all this on the windows platform!). It'd be trivial to change the z2c.recipe.zope2filestorage recipe to use a different directory. We probably need another "top level" directory though, because the part name is used as the directory name. Again, we just need a steer on what's recommended practice. For example, we could use ${buildout_home}/var/${part_name}... Martin
On 23 Jan 2007, at 23:56 , Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
Great.
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
gocept's Zope 3 instance recipe actually installs a script into the top-level bin directory, so you can do: $ bin/buildout ... $ bin/instance fg Where 'instance' is the name of the instance configuration section in buildout.cfg. So, if you have multiple instances, you can start them all with according scripts from 'bin'. It'd be really cool if z2c.recipe.zope2instance would do the same. To be frank, I think having to mess around in 'parts' sucks. I would even argue that app data like logs, databases, etc. shouldn't be in 'parts' either. You should be able to delete 'parts' and lose nothing (disclaimer: this is my view, not sure how "Buildout Jim" see it ;)).
I would tend to agree, especially since buildout has a tendency to delete things in parts/ in any case (or rather, recipes do).
I don't think it'd be hard to make such a script (perhaps more difficult to make it be cross-platform, we really need a champion for all this on the windows platform!).
It'd be trivial to change the z2c.recipe.zope2filestorage recipe to use a different directory. We probably need another "top level" directory though, because the part name is used as the directory name. Again, we just need a steer on what's recommended practice. For example, we could use ${buildout_home}/var/${part_name}...
Right. What I'm saying is that this should be the default. Sensible defaults is sometimes all it takes to get something adopted. Just look at that Plone thang ;). On another tangent, I'd like to direct your attention to grokproject (http://cheeseshop.python.org/pypi/grokproject). It's an idiot-proof way of setting up new buildouts that have grok and a custom development package preconfigured. It uses paste.script to create a raw buildout directory with a bunch of default and boilerplate things. It then bootstraps the buildout and builds the buildout. It's not rocket science, but it's made the whole "how do I get started with grok" thing a lot easier. I could envision that buildout-based deployment for end users (who don't necessarily tweak buildout.cfg etc.) could look a lot like that. Perhaps it's worth exploring this in a general manner, so that grok, Plone, and other zc.buildout consumers could share the same platform for end-user installation. I see some common goals to tackle, for example: - off-line installation (bootstrapping a buildout from already packaged eggs instead of downloading from the internet) - a Windows installer - ... Philipp
Right. What I'm saying is that this should be the default. Sensible defaults is sometimes all it takes to get something adopted. Just look at that Plone thang ;).
Yeah. I'd be happy to move the Data.fs directory to var/${part_name} under the main buildout directory. I'd also be happy to make a script in bin/ in the main buildout directory to start the instance, but I am not sure what to do about Windows. Maybe make a batchfile as well that calls runzope.bat?
On another tangent, I'd like to direct your attention to grokproject (http://cheeseshop.python.org/pypi/grokproject). It's an idiot-proof way of setting up new buildouts that have grok and a custom development package preconfigured. It uses paste.script to create a raw buildout directory with a bunch of default and boilerplate things. It then bootstraps the buildout and builds the buildout. It's not rocket science, but it's made the whole "how do I get started with grok" thing a lot easier.
Interesting. I like that idea, and we should definitely consider using PasteScript to create ploneout-like instances once we stabilise the Ploneout layout. For those who don't follow, that'd mean you'd do: $ easy_install ZopeSkel ZopeSkel contains various pastescript skeletons for zope and plone development; we could use a different package of course, but having them all in one makes sense to me. It will install PasteScript etc as well $ paster create -t plone_project And that asks you a bunch of questions and you get a new buildout.
I could envision that buildout-based deployment for end users (who don't necessarily tweak buildout.cfg etc.) could look a lot like that. Perhaps it's worth exploring this in a general manner, so that grok, Plone, and other zc.buildout consumers could share the same platform for end-user installation.
Indeed. Though to be fair, I'm not sure how much there would be to share, if you basically have a PasteScript template that creates a buildout.cfg and the buildout bootstrap.
I see some common goals to tackle, for example:
- off-line installation (bootstrapping a buildout from already packaged eggs instead of downloading from the internet)
Yes. One thing pretty high on my list would be a version of the Zope 2 installation that downloaded a tarball rather than from svn (the svn checkout is slooooooow) and/or could use an existing tarball or even an existing zope2 installation.
- a Windows installer
Yeah. As I said before, we need someone to own this on Windows. Martin
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently. the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv? currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes. for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install. as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies. -w -- ------ d. whit morriss ------ - senior engineer, opencore - - http://www.openplans.org - - m: 415-710-8975 - "If you don't know where you are, you don't know anything at all" Dr. Edgar Spencer, Ph.D., 1995
whit wrote:
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community. I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently.
I can't really answer, but it seems to be quite similar but more tightly controlled. You may find the doctests in zc.buildout (http://svn.zope.org/zc.buildout/trunk/) and zc.recipe.egg (same svn location) interesting reading, e.g. easy_install.txt.
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
As I've been talking to Whit about off-list, I find this "jail" somewhat uncomfortable. He disagrees. :)
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv?
currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes.
The way I'd sell it is that buildout lets you define a particular "build" that is to be used for repeatable deployments. Again, I'm not sure I quite see why the two can't be orthogonal in real use, e.g. the global python envionment is available to something inside a buildout even if it's come from workingenv (i.e. it wouldn't care).
for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install.
I don't see why you couldn't? If a zope instance was built with zc.buildout why wouldn't it also find something "global" (but actually local provided by workingenv)? And put differently, how does workingenv help you solve the "what is the set of packages I need to deploy my application properly and how do I set it up in a repeatable way" (which includes not only eggs but also data and configuration).
as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
I would've thought that'd work. Not sure about aggregation, but recipies are very simple and deployed as eggs (you refer to it by name, buildout finds it in pypi or your path). Martin
whit wrote:
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently.
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout.
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv?
I see no problem with that. zc.buildout is one way of deploying Python software like Zope. As long as you've got eggs, you could install them manually into your workingenv just fine. buildout just does it an automated fashion (and, of course, it can do more than just installing eggs).
currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes.
for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install.
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is.
as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;) -- http://worldcookery.com -- Professional Zope documentation and training Next Zope 3 training at Camp5: http://trizpug.org/boot-camp/camp5
Philipp von Weitershausen wrote:
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
At least it's what I want. That is, easy_install may as well put things in the ether as far as I'm concerned, and I'm more comfortable with a single file (buildout.cfg) that gives me an overview of which eggs my application consists of. Very open to be persuaded otherwise, though. ;-)
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout.
On the one hand, this patching is a bit odd, but probably just a result of the fact that Zope itself isn't terribly egg/pythonpath friendly. On the other hand, I don't like the stateful nature of the activate dance to point where it feels hacky to me. I know others disagree, though.
I see no problem with that. zc.buildout is one way of deploying Python software like Zope. As long as you've got eggs, you could install them manually into your workingenv just fine. buildout just does it an automated fashion (and, of course, it can do more than just installing eggs).
... and I've come to like its approach to stringing together recipes and passing options. It fits my brain. :)
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is.
in buildout.cfg, I did: [buildout] eggs-directory = lib/python develop-eggs-directory = lib/python Eggs are now in ${buildout}/lib/python, and it seems to work fine (but I had to stop short of testing in detail). I imagine that if workingenv was told of the same directory, the two would co-exist. I want to play with the zope 2 recipies to make filesystem layout a bit more flexible in these.
as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
Plone stinks! Martin
On Thu, 25 Jan 2007 12:45:26 -0800, Martin Aspeli <optilude@gmx.net> wrote:
Plone stinks!
It's like a fine cheese. -- Alexander Limi · http://limi.net
Philipp von Weitershausen wrote:
whit wrote:
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently.
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
honestly, it seems to me that buildout tries to do too much. it's trying to handle both repeatable deployment recipes AND providing a sandbox within which to run things. there may not be a point to having an extra layer on top of buildout, but buildout sure does seem to me a bit heavy if all i want is a sandbox. so now i have to learn the workingenv way if i just need a sandbox, but i have to learn the buildout way if i also want repeatable deployments?
Workingenv made it more complex than it needed to be (or buildout as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
this is shortsighted, IMO. i know zope quite well, but i bounced off of buildout b/c it required too much knowledge to even get started. i think it's much more likely that people from the greater python community will pick up and start using workingenv than they will zc.buildout. personally, i like chris mcdonough's approach with his buildit package. it does two things: - it retrieves files from anywhere on the 'net (cvs, svn, tarball, whatever) and puts them where you want them on your target machine(s) - it launches external scripts that then perform whatever final configuration you may want to perform. buildit is also recipe driven, and it's smart enough to pick up where it left off on a previous run, a'la make. i'd guess that you could use buildit and workingenv to accomplish pretty much everything you can do w/ zc.buildout, and you wouldn't have to bend your brain so much to do so. -r
(sorry if this ends up going out twice, I'm trying to figure out this gmane thing and apparently I hadn't figured this out when I first sent it) Whit pointed me to this thread. I won't reply to specifics here, but maybe just describe what we're doing (and planning to do), and how workingenv differs from zc.buildout. So... recently we (The Open Planning Project -- more specifically Rob Miller) decided we need a better way to deploy our specific stack. It was hard, we couldn't tell other people how to do it, and we spent a significant amount of time debugging each other's installations, or chasing down bugs that we ultimately realized were just because of slightly different installations. I'm sure you all understand the motivation; it's similar to what buildout is for, though we were more focused on repeatability as a development tool than as a deployment tool. I actually tried to do this once before with zc.buildout, but I didn't get far -- probably a result of lack of effort and lack of familiarity with the overall stack. But I also recognize lots of the questions about stuff like the zope.conf file and Data.fs that still seem unresolved. The thing that frustrated me with zc.buildout is that I knew how to do what I wanted to do, but I still felt like I was a long way from being able to do it. And little things, like unhelpful error messages and frustratingly slow performance really killed my motivation. After setting that project aside someone else at TOPP (Luke Tucker) did a buildout for Deliverance because we needed to build some non-Python libraries and that was a feature of buildout; that did end up working eventually (after considerable effort), but it was not a very satisfying experience, and *using* the buildout was itself a real challenge. Since Deliverance is just a library, to do anything useful I also had to install another package that *uses* that library, and that was surprisingly difficult. Actually developing those libraries was even more frustrating. So then Rob decides to devote some time to deployment. And because Rob just wants to get this finished, he wrote the whole things from scratch. You can see the result of that here: https://svn.openplans.org/svn/topp.deploy/trunk -- it's not a generic setup tool, just what works for us. The end result is something that does everything we want, that we understand, and that we'll understand how to extend. An important difference from zc.buildout is that all the logic and work is in *our* script, not in a framework driven by a config file. In the future Rob plans to use buildit (http://cvs.plope.com/viewcvs/Packages/buildit/), and one of the motivations there is that it has handy routines to do what topp.deploy already does, but better. It's more like a library and less like a framework. I could make some generalization here but I'll restrain myself. The deployment script also uses workingenv, somewhat similarly because it is more library-like. Well, also because I work at TOPP and can easily support their use, which is certainly a nontrivial reason for the choice. Workingenv in this case is basically a tool that provides the isolation that we need to create something repeatable. This is the main feature overlap with buildout. Workingenv is not the framework in which topp.deploy is written, and workingenv is not intended as a framework. Note also that topp.deploy does not have the full set of features we'll ultimately need. You can't tell it to install another egg, or setup another script, or whatever. And we don't *have* to add those features, because workingenv is compatible with all the other tools. Where "all the other" is mostly easy_install. But someday there will be more, even if the progress is slow. buildout is basically incompatible with easy_install (the script). And frankly I like easy_install. It's probably 10x faster than buildout. easy_install is what people use in documentation, and its conventions are the ones people know (why does buildout not use "Pkg==version", for instance?). As for the technical reasons they don't work together: * workingenv allows and leaves it to setuptools to maintain the package installation database (basically easy-install.pth). This is not a very good database, but eh. buildout doesn't really have a database, but instead just enforces what buildout.cfg indicates. * workingenv relies on that database to give default versions and to setup the Python path. The fixup it does of installed scripts is fairly minimal, just setting up sys.path enough to force its site.py to get called. buildout enumerates all the activated packages, and ignores easy-install.pth. This is basically what makes it easy_install-incompatible. Plus buildout's desire to own everything and destroy everything it does not own ;) * As a result buildout supports multiple things in the same buildout that have conflicting version requirements, but where the packages themselves don't realize this (but the deployer does). If the packages know their requirements then setuptools' native machinery allows things to work fine. The solution with workingenv is to create multiple environments. Since the actual building happens higher up (e.g., topp.deploy), there's nothing stopping you from creating multiple environments from one deployment script. Anyway, in summary the way scripts are generated is one of the major incompatibilities between buildout and workingenv. In effect Buildout's jail is too strong for workingenv to penetrate, and buildout doesn't tell anyone else about what it is doing. * workingenv allows you to change versions without informing or using workingenv. Once you've created the environment you mostly stop using workingenv directly. * Some see bin/activate as a jail. Both workingenv and buildout are deliberately jail-like. Both Jim and I loathe the non-repeatability of system-wide installations (at least I think I can speak for him on that one point ;). bin/activate lets you into that jail, and lets you work there. There is no way into a buildout. Frankly this weirds me out, and is a big part of my past frustration with it. Maybe that's because I'm in the relatively uncommon situation that I actually know what's going on under the hood of Python imports and packaging, and so it bothers me that I can't debug things directly. Anyway, neither requires activation when using scripts generated in the environment. And bin/activate is really just something that sets PYTHONPATH and then does other non-essential things like changing the prompt and $PATH -- I should probably document that more clearly. Neither can be entirely compatible with a system-wide Python installation, because Python's standard site.py f**ks up the environment really early in the process, and avoiding that isn't all that easy. In some ways virtual-python is the more complete solution to all of this, and sometimes I think I should just use that technique (of a separate Python interpreter, and hence separate prefix) with some of the improvements I could take from workingenv. Anyway, this is my very long summary of why we aren't using buildout, and are using workingenv. Cheers, Ian
Ian Bicking wrote:
After setting that project aside someone else at TOPP (Luke Tucker) did a buildout for Deliverance because we needed to build some non-Python libraries and that was a feature of buildout; that did end up working eventually (after considerable effort), but it was not a very satisfying experience, and *using* the buildout was itself a real challenge. Since Deliverance is just a library, to do anything useful I also had to install another package that *uses* that library, and that was surprisingly difficult. Actually developing those libraries was even more frustrating.
To my mind, using buildout to manage libraries like that is not what it's meant for (certainly not where I'd reach for it). Why isn't deliverance just an egg with its own dependencies? Buildout to me is a way of setting up my zope instance (which is not an egg, but a bundle of software) and pulling in the necessary eggs and products and configuration in a repeatable and mangable way. It's also fairly open-ended, so that I could make extensions to the process of setting it all up specific to my use case (by making a new recipie and wiring it into buildout.cfg, which is as simple as writing a new egg with an entry point and a single class from what I've seen so far).
So then Rob decides to devote some time to deployment. And because Rob just wants to get this finished, he wrote the whole things from scratch. You can see the result of that here: https://svn.openplans.org/svn/topp.deploy/trunk -- it's not a generic setup tool, just what works for us. The end result is something that does everything we want, that we understand, and that we'll understand how to extend. An important difference from zc.buildout is that all the logic and work is in *our* script, not in a framework driven by a config file.
I think this is fine for your own needs. But consider that we want to document how people should set up their Plone development environment. If that involves "first, write a shell script that sets it all up, a bit like so, but tailored to your own needs" then that's pretty much a dead end for a large number of people who just want to Get Things Done. At least with ploneout as it is now, you run two commands (boostrap, then buildout) and you have a fully functional Plone 3 site configured and you have something you can extend (in ways that are quite obvious to me, at least) for your own projects. However, you *didn't* have ploneout, and I suspect that starting from the zc.* recipes for Zope 3 would've been frustrating. I also suspect that trying to solve the topp.* problem without a generic Plone version to build on would've been difficult.
The deployment script also uses workingenv, somewhat similarly because it is more library-like. Well, also because I work at TOPP and can easily support their use, which is certainly a nontrivial reason for the choice. Workingenv in this case is basically a tool that provides the isolation that we need to create something repeatable. This is the main feature overlap with buildout. Workingenv is not the framework in which topp.deploy is written, and workingenv is not intended as a framework.
Indeed, and this is an important distinction.
Note also that topp.deploy does not have the full set of features we'll ultimately need. You can't tell it to install another egg, or setup another script, or whatever. And we don't *have* to add those features, because workingenv is compatible with all the other tools. Where "all the other" is mostly easy_install. But someday there will be more, even if the progress is slow. buildout is basically incompatible with easy_install (the script). And frankly I like easy_install. It's probably 10x faster than buildout.
I think buildout is mostly slow (at least from ploneout experience) because some of the generalisations are a bit optimistic. For example, the current zope 2 recipie tries to check it all out from svn, which takes forever. I plan to fix that quickly. When I run it in offline mode, it takes seconds here at least.
easy_install is what people use in documentation, and its conventions are the ones people know (why does buildout not use "Pkg==version", for instance?).
No idea, and good point. I also think it's true that people document easy_install. However, easy_install is inherently "global" (it deals with a global site-packages). workingenv "localises" this so that it tricks easy_install into thinking the instance-specific setup is the global one. It does so by fiddling environment variables. It is stateful in that you need to activate and deactivate it. To me, this is less elegant in relation to the use case of managing a particular project (what if I forget to activate and then easy_install and accidentally upgrade something...) than having a well-defined single file that explains what this project is. To some extent, working on Zope 2 projects is a bit different from working in a plain-python environment. Zope has the notion of instances with separate config files, start/stop scripts and datastore files. It's not just a script you write that sits in a package and has some dependencies. Maybe I'm blinded, though.
As for the technical reasons they don't work together:
* workingenv allows and leaves it to setuptools to maintain the package installation database (basically easy-install.pth). This is not a very good database, but eh. buildout doesn't really have a database, but instead just enforces what buildout.cfg indicates.
* workingenv relies on that database to give default versions and to setup the Python path. The fixup it does of installed scripts is fairly minimal, just setting up sys.path enough to force its site.py to get called. buildout enumerates all the activated packages, and ignores easy-install.pth. This is basically what makes it easy_install-incompatible. Plus buildout's desire to own everything and destroy everything it does not own ;)
That is somewhat down to recipes, though. I certainly want to tie this down in the zope 2 recipies to be less destructive (we're already almost there).
* As a result buildout supports multiple things in the same buildout that have conflicting version requirements, but where the packages themselves don't realize this (but the deployer does). If the packages know their requirements then setuptools' native machinery allows things to work fine. The solution with workingenv is to create multiple environments. Since the actual building happens higher up (e.g., topp.deploy), there's nothing stopping you from creating multiple environments from one deployment script. Anyway, in summary the way scripts are generated is one of the major incompatibilities between buildout and workingenv. In effect Buildout's jail is too strong for workingenv to penetrate, and buildout doesn't tell anyone else about what it is doing.
* workingenv allows you to change versions without informing or using workingenv. Once you've created the environment you mostly stop using workingenv directly.
* Some see bin/activate as a jail. Both workingenv and buildout are deliberately jail-like. Both Jim and I loathe the non-repeatability of system-wide installations (at least I think I can speak for him on that one point ;). bin/activate lets you into that jail, and lets you work there. There is no way into a buildout. Frankly this weirds me out, and is a big part of my past frustration with it. Maybe that's because I'm in the relatively uncommon situation that I actually know what's going on under the hood of Python imports and packaging, and so it bothers me that I can't debug things directly.
Probably. I think it also has something to do with the fact that Zope people (me included) think in the context of a Zope instance as the ultimate controller of what gets executed. The buildout builds an instance plus some other stuff and informs it of what packages it has available.
Anyway, neither requires activation when using scripts generated in the environment. And bin/activate is really just something that sets PYTHONPATH and then does other non-essential things like changing the prompt and $PATH -- I should probably document that more clearly. Neither can be entirely compatible with a system-wide Python installation, because Python's standard site.py f**ks up the environment really early in the process, and avoiding that isn't all that easy. In some ways virtual-python is the more complete solution to all of this, and sometimes I think I should just use that technique (of a separate Python interpreter, and hence separate prefix) with some of the improvements I could take from workingenv.
Anyway, this is my very long summary of why we aren't using buildout, and are using workingenv.
I am still trying to learn more about the details of how eggs and dependency management work. You certainly understand it better than me, and I found some of this quite enlightening. I would love it if Jim or someone more involved in the original design of zc.buildout had anything to add in answer to your specific points above. :) Martin
Rob Miller wrote:
honestly, it seems to me that buildout tries to do too much. it's trying to handle both repeatable deployment recipes AND providing a sandbox within which to run things. there may not be a point to having an extra layer on top of buildout, but buildout sure does seem to me a bit heavy if all i want is a sandbox. so now i have to learn the workingenv way if i just need a sandbox, but i have to learn the buildout way if i also want repeatable deployments?
Potentially. But I find it kind of reassuring to have a well-defined list of which eggs are part of my "special" environment i.e. the one tied to my instance.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
this is shortsighted, IMO. i know zope quite well, but i bounced off of buildout b/c it required too much knowledge to even get started. i think it's much more likely that people from the greater python community will pick up and start using workingenv than they will zc.buildout.
Again, I think the two are orthogonal. And honestly, I found zc.buildout pretty easy to understand. I resisted it for a while, it seems liked it *should* be complex, and I won't pretend to understand how it manages eggs in any detail, but I don't think it matters. Look at http://dev.plone.org/plone/browser/ploneout/trunk/buildout.cfg - I find that pretty self-explanatory. I tried to document how it works at a high level and how you may use it here http://dev.plone.org/plone/browser/ploneout/trunk/README.txt. And writing a new recipe is as simple as this: http://dev.plone.org/plone/browser/ploneout/trunk/src/z2c.recipe.zope2checko... All that is plain python code, the only thing you need to understand about buildout is that it has a dict-like object called 'options' which reflects the options in the current part's section in buildout.cfg, and a higher level dict-like object called 'buildout' which has the options for all the parts (so buildout['foo'] are the options for part [foo] in buidout.cfg). Each part is associated with a recipie. Recipies are ordered.
personally, i like chris mcdonough's approach with his buildit package. it does two things:
- it retrieves files from anywhere on the 'net (cvs, svn, tarball, whatever) and puts them where you want them on your target machine(s)
You can do that quite easily with buildout as well. I would like to make a more generic recipe for retrieving tarballs for e.g. zope installation, and I think it'd be as hard as figuring out which python function to use to download something.
- it launches external scripts that then perform whatever final configuration you may want to perform.
Again, if you want a recipe to do that, it's trivial to write (10 lines of code?). Instead of an external script, though, I would probably find it easier to write that as a buildout recipe in python.
buildit is also recipe driven, and it's smart enough to pick up where it left off on a previous run, a'la make. i'd guess that you could use buildit and workingenv to accomplish pretty much everything you can do w/ zc.buildout, and you wouldn't have to bend your brain so much to do so.
I'm just struggling to see why it's so hard to figure out how buildout works. Perhaps it just fits my brain. But honestly, once Hanno showed me his initial steps with ploneout and I'd taken ten minutes with pdb inside the __init__() and install() functions (the only two...) in his recipe the pieces fitted together in my head almost instantly. I don't greatly care how the standard zc.recipe.egg mechanism installs eggs because it works and because I can see clearly where they come from, how I create new ones and I'm satisfied that they are available ot my python interpreter. If I did care, I'm sure I wouldn't find it that hard to trace, though. Martin
Martin Aspeli wrote:
Rob Miller wrote:
honestly, it seems to me that buildout tries to do too much. it's trying to handle both repeatable deployment recipes AND providing a sandbox within which to run things. there may not be a point to having an extra layer on top of buildout, but buildout sure does seem to me a bit heavy if all i want is a sandbox. so now i have to learn the workingenv way if i just need a sandbox, but i have to learn the buildout way if i also want repeatable deployments?
Potentially. But I find it kind of reassuring to have a well-defined list of which eggs are part of my "special" environment i.e. the one tied to my instance.
to find this i simply look in /path/to/workingenv/lib/python2.4.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
this is shortsighted, IMO. i know zope quite well, but i bounced off of buildout b/c it required too much knowledge to even get started. i think it's much more likely that people from the greater python community will pick up and start using workingenv than they will zc.buildout.
Again, I think the two are orthogonal.
orthogonal, but with overlap.
And honestly, I found zc.buildout pretty easy to understand. I resisted it for a while, it seems liked it *should* be complex, and I won't pretend to understand how it manages eggs in any detail, but I don't think it matters.
Look at http://dev.plone.org/plone/browser/ploneout/trunk/buildout.cfg - I find that pretty self-explanatory. I tried to document how it works at a high level and how you may use it here http://dev.plone.org/plone/browser/ploneout/trunk/README.txt.
And writing a new recipe is as simple as this:
http://dev.plone.org/plone/browser/ploneout/trunk/src/z2c.recipe.zope2checko...
All that is plain python code, the only thing you need to understand about buildout is that it has a dict-like object called 'options' which reflects the options in the current part's section in buildout.cfg, and a higher level dict-like object called 'buildout' which has the options for all the parts (so buildout['foo'] are the options for part [foo] in buidout.cfg). Each part is associated with a recipie. Recipies are ordered.
personally, i like chris mcdonough's approach with his buildit package. it does two things:
- it retrieves files from anywhere on the 'net (cvs, svn, tarball, whatever) and puts them where you want them on your target machine(s)
You can do that quite easily with buildout as well. I would like to make a more generic recipe for retrieving tarballs for e.g. zope installation, and I think it'd be as hard as figuring out which python function to use to download something.
i have no doubt that zc.buildout will do everything that buildit will do, and probably much more. but i like simple. and for me, having something light like workingenv to manage my sandbox, and a library like buildit for putting files into those sandboxes feels simple.
- it launches external scripts that then perform whatever final configuration you may want to perform.
Again, if you want a recipe to do that, it's trivial to write (10 lines of code?). Instead of an external script, though, I would probably find it easier to write that as a buildout recipe in python.
the script could of course be python as well.
buildit is also recipe driven, and it's smart enough to pick up where it left off on a previous run, a'la make. i'd guess that you could use buildit and workingenv to accomplish pretty much everything you can do w/ zc.buildout, and you wouldn't have to bend your brain so much to do so.
I'm just struggling to see why it's so hard to figure out how buildout works. Perhaps it just fits my brain. But honestly, once Hanno showed me his initial steps with ploneout and I'd taken ten minutes with pdb inside the __init__() and install() functions (the only two...) in his recipe the pieces fitted together in my head almost instantly.
your efforts to figure something out and then document will have a major impact on people understanding this, as per usual. but still, something about it just feels heavy. the idea of having two separate tools, each of which does one thing well, working together to solve a problem suits my sensibilities more than having one more monolithic tool. luckily, we don't really have to convince each other of anything, here. it's entirely possible to have zc.buildout recipes for installing Zope and Plone as well as buildit / workingenv recipes for the same purpose. -r
Rob Miller wrote:
honestly, it seems to me that buildout tries to do too much.
That's ok. I often don't need the big hammer that buildout is. That's when I tend to use workingenv (even if it's' just for trying out whether something's easy_install'able)
it's trying to handle both repeatable deployment recipes AND providing a sandbox within which to run things. there may not be a point to having an extra layer on top of buildout, but buildout sure does seem to me a bit heavy if all i want is a sandbox.
Buildout is actually quite simple, it took me a while to grok it, too. I started writing a tutorial for it but got interupted by other stuff. I sure hope to finish it.
so now i have to learn the workingenv way if i just need a sandbox, but i have to learn the buildout way if i also want repeatable deployments?
I think buildout can serve well for sandboxes if you work in a team, because everybody uses the same recipe for sandboxes.
Workingenv made it more complex than it needed to be (or buildout as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
this is shortsighted, IMO. i know zope quite well, but i bounced off of buildout b/c it required too much knowledge to even get started.
There's nothing Zope specific about buildout. It just happens to be developed by Zope people. The egg and cmmi recipes allow it to be used for the majority of other Python software out there, though (and if that's not enough, you can alway simplement a custom recipe). Note that I bounced off of it too, until Martijn and Christian explained it to me and it was forced on me via grok. It's purely a documentation issue, I think.
personally, i like chris mcdonough's approach with his buildit package. it does two things:
- it retrieves files from anywhere on the 'net (cvs, svn, tarball, whatever) and puts them where you want them on your target machine(s)
- it launches external scripts that then perform whatever final configuration you may want to perform.
buildit is also recipe driven, and it's smart enough to pick up where it left off on a previous run, a'la make. i'd guess that you could use buildit and workingenv to accomplish pretty much everything you can do w/ zc.buildout, and you wouldn't have to bend your brain so much to do so.
Sure. My point has always been that everybody should choose the tools they understand best. I think there's a place for both workingenv and buildout out there. I don't think that's short-sighted. My only assertion was that using workingenv and buildout together seems like overcomplicating things, as they're both trying to solve a lot of similar issues. -- http://worldcookery.com -- Professional Zope documentation and training Next Zope 3 training at Camp5: http://trizpug.org/boot-camp/camp5
Philipp von Weitershausen wrote:
whit wrote:
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently.
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
well that's awesome that buildout has duplicated this effort....
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout. is there really a difference between zopectl and source bin/activate? I guess the main difference here is where PYTHONPATH gets set and how people work with it. for example, if you just start python and want to play with code.... sounds like with zc.buildout you are out of luck for things installed in the buildout.
In our situation, we might have multiple versions of software running on different processes started in different workingenv (often not even using zope). being able to contextualize the python path is a useful development tool; this is understandable for buildout to avoid(as a deployment tool), but if we are considering using buildout as a prescribed way for people to manage their development environments, it seems lacking.
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv?
I see no problem with that. zc.buildout is one way of deploying Python software like Zope. As long as you've got eggs, you could install them manually into your workingenv just fine. buildout just does it an automated fashion (and, of course, it can do more than just installing eggs).
which I could automate in my env (workingenv takes recipes also). for that matter, eggs automate the installation of eggs... I'm not sure what it would take to make buildout just automate install eggs into my env... but not putting them in special directories would be a start.
currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes.
for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install.
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is. +1
this may be all that is required to make the two play well together(or have buildout respect an existing workingenv when doing it's local install of eggs). Maybe it's just a matter of zc.buildout seeing workingenv is in effect and not composing special pythonpaths. harmony is my interest here. workingenv is pretty low-level and works hard to work well with python. there is no reason that zc.buildout should have a problem with that.. it just needs to do less when appropriate.
as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout-squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
ooooooh philip... <rant> you're drawing my ire for taking a stereotypical stone-age isolationist zope developer attitude toward the rest of the world. shame on you(though I don't think you really mean this. so take this as feedback). who did I say thinks zope smells? my developers(the people I have manage and work with)... who as non-zope python programmers are rightfully a little leery of zope kool-aid and whether it's worth their time, and like using workingenv as a python development environment. Telling them that there is no need for workingenv is not acceptable and they should just use new way X because some really smart zope guys think it's great really doesn't fly; they are already happily using it and just want to install stuff they need into where they are working now. I get really tired of defending zope world tech and things like this *don't* help. you can't build bridges by telling people to burn their old ones. let's put this way, in the democratic hierarchy that is my work place, I have to justify zope and every other technology we use and these are the kinds of battles that loses the zope community developers. try to justify why a zope hammer is better and why zope folk keep building new hammers that don't play nice with everyone else's hammer loses us cred. Credibility is something my job depends on as does the job of everyone who has to make technology decisions and has to stand by them. Pointing at zc.buildout and saying it handles eggs and i should just rebuild the basis of all the deployments ten developers are working happily and effectively on makes me look like an idiot. next time we decide whether to write a zope app or something else, I'm probably not going to have much of a say. </rant> -w -- ------ d. whit morriss ------ - senior engineer, opencore - - http://www.openplans.org - - m: 415-710-8975 - "If you don't know where you are, you don't know anything at all" Dr. Edgar Spencer, Ph.D., 1995
whit wrote:
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout. is there really a difference between zopectl and source bin/activate? I guess the main difference here is where PYTHONPATH gets set and how people work with it. for example, if you just start python and want to play with code.... sounds like with zc.buildout you are out of luck for things installed in the buildout.
In our situation, we might have multiple versions of software running on different processes started in different workingenv (often not even using zope). being able to contextualize the python path is a useful development tool; this is understandable for buildout to avoid(as a deployment tool), but if we are considering using buildout as a prescribed way for people to manage their development environments, it seems lacking.
I think this is the basic difference -- workingenv is development-centric, while buildout is deployment-centric. This does not necessarily mean "the best tool for the job", because focusing on development and ignore deployment isn't a good job, nor the other way around. At TOPP we know how to address both stories with workingenv. We haven't figured it out with buildout, despite trying. And we definitely aren't satisfied with just a deployment tool that doesn't address our development needs. But there's a lot of use cases for Plone specifically where carefully developing just a deployment solution makes sense. That doesn't make sense for us, because we're not in the kind of consulting business where the relative scale of development and deployment works like that. But eh; the presence of a buildout for Plone doesn't hurt our position any. Anything that gets rid of another ad hoc crappy deployment is good by me. And if other people are working on it, all the better! If *Plone* becomes incompatible with workingenv that'd be bothersome (though it was not compatible with workingenv without some work in the first place). But if a buildout is incompatible, eh... who knows, it might even make sense to create something like a "freeze this workingenv as a buildout" script. That one directory structure can't be both at the same time isn't a huge problem in my mind.
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is. +1
this may be all that is required to make the two play well together(or have buildout respect an existing workingenv when doing it's local install of eggs). Maybe it's just a matter of zc.buildout seeing workingenv is in effect and not composing special pythonpaths.
harmony is my interest here. workingenv is pretty low-level and works hard to work well with python. there is no reason that zc.buildout should have a problem with that.. it just needs to do less when appropriate.
Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy. I'd rather see easy-install.pth become a better package database, or some other strategy of external requirement specifications, instead of building it into scripts. I understand the issue Jim is trying to solve here, but putting everything into the buildout.cfg and then imperatively setting up the files from there bothers me and does not make development easy IMHO. That's mostly the problem. Then workingenv would do its part by monkeypatching distutils and setuptools to install things locally, and changing site.py to not automatically pick up things globally. And buildout could do its stuff that applies to other more complicated setups than just setting up some libraries, which is about where workingenv stops. Well, what I'd *really* like is an idea of a working environment that applies more widely than Python, because other languages (including but not limited to the dynamic languages) have all these same problems, and we all have half-assed solutions. But I don't even know how to start that conversation. And don't get me started on how the distributions are going to look at all this! Ian
On Jan 25, 2007, at 5:44 PM, Ian Bicking wrote:
workingenv is development-centric, while buildout is deployment- centric. This does not necessarily mean "the best tool for the job", because focusing on development and ignore deployment isn't a good job, nor the other way around.
buildout focusses on both. ...
If *Plone* becomes incompatible with workingenv that'd be bothersome
I agree.
But if a buildout is incompatible, eh... who knows,
I would hope that buildout would not have to be compatible with workingenv, whatever that means, in order for Plone to be compatible. Then again, I'm not sure what compatibility means in this context.
it might even make sense to create something like a "freeze this workingenv as a buildout" script. That one directory structure can't be both at the same time isn't a huge problem in my mind.
Deployment involves far more than getting the software installed. ...
Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy.
Um, we're both changing the path -- just in different ways.
I'd rather see easy-install.pth become a better package database, or some other strategy of external requirement specifications, instead of building it into scripts.
We certainly disagree there. OTOH, I wouldn't be opposed to having a recipe for generating scripts that used the .pth files created by workingenv.
I understand the issue Jim is trying to solve here, but putting everything into the buildout.cfg and then imperatively setting up the files from there bothers me and does not make development easy IMHO.
This seems to be a matter of taste. I like developing with buildout. It makes development easier for me. :) To each his own.
That's mostly the problem. Then workingenv would do its part by monkeypatching distutils and setuptools to install things locally, and changing site.py to not automatically pick up things globally.
I applaud you for this effort. I chose to work below easy_install and site.py rather that trying to bend/monkey-path it to my will. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
If *Plone* becomes incompatible with workingenv that'd be bothersome
I agree.
But if a buildout is incompatible, eh... who knows,
I would hope that buildout would not have to be compatible with workingenv, whatever that means, in order for Plone to be compatible. Then again, I'm not sure what compatibility means in this context.
It would be a concern if, for instance, Plone started depending on buildout recipes for installation, without "plain" distutils recipes. Of course right now there are no distutils recipes for old-style Products. So actually it's an active issue -- if buildout enables Plone to keep from moving to distutils/setuptools/egg-style package deployment (because buildout is flexible enough to support the old patterns) then that would make it harder for workingenv. But I don't think anyone is proposing that buildout means that Plone (and Zope 2/Five) shouldn't continue its move towards traditional Python packaging. So basically I would just hope that Plone doesn't lean to heavily on buildout.
it might even make sense to create something like a "freeze this workingenv as a buildout" script. That one directory structure can't be both at the same time isn't a huge problem in my mind.
Deployment involves far more than getting the software installed.
Yes; in a workingenv model you start with an environment, then you actually make your deployment using tools of your choice. workingenv doesn't deploy for you. Admittedly "your choice" isn't always good, because maybe you didn't want to make a choice ;)
Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy.
Um, we're both changing the path -- just in different ways.
Maybe "enumeration vs adding a directory of eggs" is a better description. Setuptools controls the directory of eggs (via easy-install.pth), while buildout controls the scripts and doesn't give setuptools that control.
I'd rather see easy-install.pth become a better package database, or some other strategy of external requirement specifications, instead of building it into scripts.
We certainly disagree there. OTOH, I wouldn't be opposed to having a recipe for generating scripts that used the .pth files created by workingenv.
I'm not sure what the problem would be? I appreciate the desire for a set of requirements that is different from the requirements of the package, and workingenv has some support for that (with the --requirements option), but it's a different style from buildout. easy-install.pth would almost be okay, except for the fact that it is constantly pointing to things that don't exist, has setuptools' hacks in it (to work around the atrocious Python standard site.py), and doesn't describe intent, it's just an enumeration of what is available and "activated" (i.e., available without specifically requiring something). Besides all that, it's still a workable foundation IMHO. -- Ian Bicking | ianb@colorstudy.com | http://blog.ianbicking.org
Ian Bicking wrote:
It would be a concern if, for instance, Plone started depending on buildout recipes for installation, without "plain" distutils recipes. Of course right now there are no distutils recipes for old-style Products. So actually it's an active issue -- if buildout enables Plone to keep from moving to distutils/setuptools/egg-style package deployment (because buildout is flexible enough to support the old patterns) then that would make it harder for workingenv. But I don't think anyone is proposing that buildout means that Plone (and Zope 2/Five) shouldn't continue its move towards traditional Python packaging.
No, quite the contrary in fact. The other day I wrote a proposal that Zope's old-style Products will turn into regular python packages using pkg_resource's namespace package support and eggs' entry points (http://wiki.zope.org/zope2/EggifyingZopesExtensionMechanismQuotProductsQuot). It will allow us to deploy old-style products as eggs (using setup.py). The proposal explicitly mentions workingenv and zc.buildout as two mechanisms that allow such deployment w/ Zope. -- http://worldcookery.com -- Professional Zope documentation and training Next Zope 3 training at Camp5: http://trizpug.org/boot-camp/camp5
Ian Bicking wrote:
Jim Fulton wrote:
If *Plone* becomes incompatible with workingenv that'd be bothersome
I agree.
But if a buildout is incompatible, eh... who knows,
I would hope that buildout would not have to be compatible with workingenv, whatever that means, in order for Plone to be compatible. Then again, I'm not sure what compatibility means in this context.
It would be a concern if, for instance, Plone started depending on buildout recipes for installation, without "plain" distutils recipes. Of course right now there are no distutils recipes for old-style Products. So actually it's an active issue -- if buildout enables Plone to keep from moving to distutils/setuptools/egg-style package deployment (because buildout is flexible enough to support the old patterns) then that would make it harder for workingenv. But I don't think anyone is proposing that buildout means that Plone (and Zope 2/Five) shouldn't continue its move towards traditional Python packaging.
So basically I would just hope that Plone doesn't lean to heavily on buildout.
Certainly buildout encourages the use of eggs. I think we're all moving toward the use of eggs pretty aggressively. Of course, a Zope installation, most notably instance creation, has requirements beyond the scope of eggs. Buildout addresses those requirements.
it might even make sense to create something like a "freeze this workingenv as a buildout" script. That one directory structure can't be both at the same time isn't a huge problem in my mind.
Deployment involves far more than getting the software installed.
Yes; in a workingenv model you start with an environment, then you actually make your deployment using tools of your choice. workingenv doesn't deploy for you. Admittedly "your choice" isn't always good, because maybe you didn't want to make a choice ;)
And buildout is one such tool.
Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy.
Um, we're both changing the path -- just in different ways.
Maybe "enumeration vs adding a directory of eggs" is a better description. Setuptools controls the directory of eggs (via easy-install.pth), while buildout controls the scripts and doesn't give setuptools that control.
No, the user controls the directory of eggs using easy_install, workingenv, or buildout. easy_install goes further by creating .pth files. A .pth contains a single static path. This is really no-different than a single static path baked into a script by buildout. In both cases, the user has some control over how this path is baked. I chose to include the path in the script because it makes the script more independent of it's environment. Once created, the script can be moved or linked anywhere and run from anywhere without any brittle rules for finding the .pth file. Phillip Eby suggested that script- dependent .pth files might be put alongside the script, but I think just including the path in the script is cleaner. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
Ian Bicking wrote:
Jim Fulton wrote:
If *Plone* becomes incompatible with workingenv that'd be bothersome I agree.
But if a buildout is incompatible, eh... who knows, I would hope that buildout would not have to be compatible with workingenv, whatever that means, in order for Plone to be compatible. Then again, I'm not sure what compatibility means in this context. It would be a concern if, for instance, Plone started depending on buildout recipes for installation, without "plain" distutils recipes. Of course right now there are no distutils recipes for old-style Products. So actually it's an active issue -- if buildout enables Plone to keep from moving to distutils/setuptools/egg-style package deployment (because buildout is flexible enough to support the old patterns) then that would make it harder for workingenv. But I don't think anyone is proposing that buildout means that Plone (and Zope 2/Five) shouldn't continue its move towards traditional Python packaging.
So basically I would just hope that Plone doesn't lean to heavily on buildout.
Certainly buildout encourages the use of eggs. I think we're all moving toward the use of eggs pretty aggressively.
Of course, a Zope installation, most notably instance creation, has requirements beyond the scope of eggs. Buildout addresses those requirements.
it might even make sense to create something like a "freeze this workingenv as a buildout" script. That one directory structure can't be both at the same time isn't a huge problem in my mind. Deployment involves far more than getting the software installed. Yes; in a workingenv model you start with an environment, then you actually make your deployment using tools of your choice. workingenv doesn't deploy for you. Admittedly "your choice" isn't always good, because maybe you didn't want to make a choice ;)
And buildout is one such tool.
Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy. Um, we're both changing the path -- just in different ways. Maybe "enumeration vs adding a directory of eggs" is a better description. Setuptools controls the directory of eggs (via easy-install.pth), while buildout controls the scripts and doesn't give setuptools that control.
No, the user controls the directory of eggs using easy_install, workingenv, or buildout.
I don't think buildout's default locations would be called "sensible" by anybody except the folks who wrote it. Here is some of what I find puzzling: - Why don't binary eggs go in one of the "standard" location (lib/python or lib/python2.x)? - Why not put development eggs in a directory which matches existing conventions ('src')? - Why are pieces of the buildout squirreled away under 'parts', instead of putting them in a location which signals their purpose ('lib', 'libexec', 'var', etc.)? - Why do I have to cd down into 'parts/instances/foo' to run the application which is the point of the buildout, rather than running scripts from 'bin'? The perplexing filesystem layout was one of the biggest strikes against zpkg: it feels as though buildout has some of the same heritage.
easy_install goes further by creating .pth files. A .pth contains a single static path. This is really no-different than a single static path baked into a script by buildout.
Except that *any* script running in that enviroment has access to the paths set up by *all* the egg installations: adding a new egg doesn't require touching the scripts.
In both cases, the user has some control over how this path is baked. I chose to include the path in the script because it makes the script more independent of it's environment. Once created, the script can be moved or linked anywhere and run from anywhere without any brittle rules for finding the .pth file. Phillip Eby suggested that script- dependent .pth files might be put alongside the script, but I think just including the path in the script is cleaner.
For deployment, perhaps, but you have to munge *each* script when you add a new egg in development, rather than letting setuptools tweak the .pth files (which are the "standard" Python path extension mechanism). It is therefore *much* harder to "try out" a new egg in a buildout environment, using easy_install / setuptools, than in a workingenv (or the environment created by 'virtual_python', which is what I use on a day-to-day basis). I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFuo01+gerLs4ltQ4RAj6cAJ4vZ0K6hHhRbLuRuXWyqT/KiVDMUwCeLLMw SVjCFguTUapLe4funr7Ov1I= =68uB -----END PGP SIGNATURE-----
Tres Seaver wrote:
I don't think buildout's default locations would be called "sensible" by anybody except the folks who wrote it.
I think a lot of this may have to do with sensible defaults; most (all?) of this is settable via options in buildout.cfg, which is reassuring at least.
Here is some of what I find puzzling:
- Why don't binary eggs go in one of the "standard" location (lib/python or lib/python2.x)?
I'd tend to agree that that'd make sense. I think for ploneout's buildout.cfg, we may set the option accordingly.
- Why not put development eggs in a directory which matches existing conventions ('src')?
At least as it's used in ploneout, the development eggs are in src/ and when they're installed, you get .egg-link files in development-eggs. We may again put that into lib/python for ploneout's purposes.
- Why are pieces of the buildout squirreled away under 'parts', instead of putting them in a location which signals their purpose ('lib', 'libexec', 'var', etc.)?
Where would a Zope 2 instance or a Zope 2 checkout sit? I thinks parts/ is as good as anything here. For the z2c.recipe.zope2filestorage (which manages the location of Data.fs) recipe I've just made it so that it uses 'var' instead of parts.
- Why do I have to cd down into 'parts/instances/foo' to run the application which is the point of the buildout, rather than running scripts from 'bin'?
Hanno is working on a recipe to let us get the zope instance start/stop script in the top level bin for ploneout. There's already a recipe for running tests from there. Philipp was of the opinion that you shouldn't have to care about anything inside parts (it's buildout's playground) and I'd like to make it so that this is the case.
In both cases, the user has some control over how this path is baked. I chose to include the path in the script because it makes the script more independent of it's environment. Once created, the script can be moved or linked anywhere and run from anywhere without any brittle rules for finding the .pth file. Phillip Eby suggested that script- dependent .pth files might be put alongside the script, but I think just including the path in the script is cleaner.
For deployment, perhaps, but you have to munge *each* script when you add a new egg in development, rather than letting setuptools tweak the .pth files (which are the "standard" Python path extension mechanism). It is therefore *much* harder to "try out" a new egg in a buildout environment, using easy_install / setuptools, than in a workingenv (or the environment created by 'virtual_python', which is what I use on a day-to-day basis).
I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation
I would agree that having to mangle multiple scripts is annoying. On the other hand, I find the activate/deactivate dance annoying. I rather think that the scripts that buildout has created should not require any special activation or introduction to a particular environment. Perhaps a solution is to let buildout put things in standard places (as I'm typing this, we now have all the ploneout .egg and .egg-link files/directories in ${buildout}/lib/python) and then have an optional workingenv at the root of the buildout that could be activated when you want that, but let the buildout-managed scripts (e.g. zope start/stop and test runner) not require this? I have a feeling there's more to this than agreeing on directory layout though. But I hope it oughtn't be *that much more*. Martin
I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation
I would agree that having to mangle multiple scripts is annoying. On the other hand, I find the activate/deactivate dance annoying. I rather think that the scripts that buildout has created should not require any special activation or introduction to a particular environment.
Perhaps a solution is to let buildout put things in standard places (as I'm typing this, we now have all the ploneout .egg and .egg-link files/directories in ${buildout}/lib/python) and then have an optional workingenv at the root of the buildout that could be activated when you want that, but let the buildout-managed scripts (e.g. zope start/stop and test runner) not require this?
I have a feeling there's more to this than agreeing on directory layout though. But I hope it oughtn't be *that much more*.
I did some experimentation with this. Basically, I did: - make egg-directory and develop-egg-directory in buildout.cfg be lib/python2.4 (which is what workingenv uses) - run the buildout again - create standard workingenv (python workingenv.py .) in the root buildout directory The results are fairly encouraging: - buildout-installed eggs don't work in the workingenv jail... - ...but they do if I manually add them to lib/python2.4/easy_install.pth - easy_installed packages (i.e. installed with workingenv's easy_install) are available to zope This was only a brief test, but seems to me that if buildout was managing the easy_install.pth file we wouldn't be a million miles off something where the source bin/activate dance was optional and would work as a generic solution, but would not be required for zope's startup and other buildout-managed scripts. However, I'm not sure whether and how buildout should manage easy_install.pth in a non-hacky way. A hacky way would be to have a buildout recipie that, if included, would just patch that file up based on buildout['buildout']['eggs'] (i.e. the list of eggs listed in buildout.cfg under the main [buildout] section). I don't really understand the purpose of easy_install.pth and why it matters here, though, so suggestions welcome. :) Martin
Martin Aspeli wrote:
I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation
I would agree that having to mangle multiple scripts is annoying. On the other hand, I find the activate/deactivate dance annoying. I rather think that the scripts that buildout has created should not require any special activation or introduction to a particular environment.
Perhaps a solution is to let buildout put things in standard places (as I'm typing this, we now have all the ploneout .egg and .egg-link files/directories in ${buildout}/lib/python) and then have an optional workingenv at the root of the buildout that could be activated when you want that, but let the buildout-managed scripts (e.g. zope start/stop and test runner) not require this?
I have a feeling there's more to this than agreeing on directory layout though. But I hope it oughtn't be *that much more*.
I did some experimentation with this. Basically, I did:
- make egg-directory and develop-egg-directory in buildout.cfg be lib/python2.4 (which is what workingenv uses)
- run the buildout again
- create standard workingenv (python workingenv.py .) in the root buildout directory
The results are fairly encouraging:
- buildout-installed eggs don't work in the workingenv jail... - ...but they do if I manually add them to lib/python2.4/easy_install.pth - easy_installed packages (i.e. installed with workingenv's easy_install) are available to zope
This was only a brief test, but seems to me that if buildout was managing the easy_install.pth file we wouldn't be a million miles off something where the source bin/activate dance was optional and would work as a generic solution, but would not be required for zope's startup and other buildout-managed scripts.
I would assume that buildout is specifically disabling easy_install's updating of easy-install.pth -- buildout is still installing packages with easy_install, but with some monkeypatches of the command. workingenv also monkeypatches easy_install, but only to change script creation (as noted in another email) and to keep it from rejecting workingenv's site.py. Anyway, if easy_install was allowed to do its thing then they'd be more compatible.
However, I'm not sure whether and how buildout should manage easy_install.pth in a non-hacky way. A hacky way would be to have a buildout recipie that, if included, would just patch that file up based on buildout['buildout']['eggs'] (i.e. the list of eggs listed in buildout.cfg under the main [buildout] section).
I don't really understand the purpose of easy_install.pth and why it matters here, though, so suggestions welcome. :)
Eggs exist to pkg_resources (the runtime portion of setuptools) simply by being available on a path. E.g., if you have lib/python/ on the path, and lib/python/Foo-1.0.egg/ exists, then if you do pkg_resources.require('Foo') that will add lib/python/Foo-1.0.egg/ to the path, and you can import from it. When you can actually import from the egg it is "active". You can also see its entry points at that time, but not if it is just available. If lib/python/Foo-1.0.egg/ is on the path to start with you can import from it directly. The eggs listed in easy-install.pth are put on the path, and hence are activated by default. You can still require a different egg version (e.g., Foo==1.1), and I believe setuptools will (if it can) switch the path to remove the default activated egg and put on the new egg. If you install a package with easy_install -m ("multiversion") then none of the versions go on the path, and eggs will only be added when they are specifically required. Any egg listed in "install_requires" in setup.py will get added as soon as you require the original egg, and when you run a script from an egg it'll activate that egg as well, so if you are thorough with your requirements then none of the eggs need to be activated. One problem if you are using entry points is that if your egg isn't activated and you are searching entry points by name (not by egg) then your egg has to be activated to work (Buffet and paster create templates are both searched by name, for example). I have expected this to cause more problems than it has, but since people have stopped leaning too heavily on setuptools versioning (thanks to virtual-python, workingenv, and buildout) I think it hasn't come up much. Anyway, if Zope was to start using entry points to replace some of its ZCML (which I personally think would be a good idea) this is something to be aware of. Ian
Ian Bicking wrote: ...
I would assume that buildout is specifically disabling easy_install's updating of easy-install.pth
WRT egg installation, buildout follows easy_install's multi-version model. It installs eggs in such a way that multiple versions can be installed at the same time. As with easy_install in multi-version mode, no easy_install.pth file is updated.
-- buildout is still installing packages with easy_install,
Not nearly as much as it used to. In fact, buildout only uses easy_install in one or 2 special cases. As I've learned more about setuptools, I've been able to use lower-level interfaces, which are much cleaner.
but with some monkeypatches of the command. workingenv also monkeypatches easy_install, but only to change script creation (as noted in another email) and to keep it from rejecting workingenv's site.py.
Anyway, if easy_install was allowed to do its thing then they'd be more compatible.
I don't think this has much to do with compatibility. The first step to compatibility is deciding what it means. :) I'm all in favor of workingenv/buildout compatibility. I'd like to see some specifics of how people would like to use workingenv amd buildout together. I have some guesses, but I'd rather hear people say what they want to do. I think this would be much more useful than a discussion of implementation details at this point. Once we know what we want the end result to be, I'm sure you and I can work out some implementation that makes sense. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
The first step to compatibility is deciding what it means. :) I'm all in favor of workingenv/buildout compatibility. I'd like to see some specifics of how people would like to use workingenv amd buildout together. I have some guesses, but I'd rather hear people say what they want to do. I think this would be much more useful than a discussion of implementation details at this point. Once we know what we want the end result to be, I'm sure you and I can work out some implementation that makes sense.
I agree, and I find myself a bit confused by this orientation as well. The main use case I could imagine wanting to solve would be that I'd like to run 'python' inside zc.buildout and have an interactive prompt that had all the eggs that zc.buildout knew about available. That is, I'd like to be able to do "from zope.interface import ..." and so on. Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment (in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them). In both these cases, I wouldn't care much about workingenv per se, only that I had a sensible way of managing this environment. Martin
Martin Aspeli wrote:
Jim Fulton wrote:
The first step to compatibility is deciding what it means. :) I'm all in favor of workingenv/buildout compatibility. I'd like to see some specifics of how people would like to use workingenv amd buildout together. I have some guesses, but I'd rather hear people say what they want to do. I think this would be much more useful than a discussion of implementation details at this point. Once we know what we want the end result to be, I'm sure you and I can work out some implementation that makes sense.
I agree, and I find myself a bit confused by this orientation as well.
The main use case I could imagine wanting to solve would be that I'd like to run 'python' inside zc.buildout and have an interactive prompt that had all the eggs that zc.buildout knew about available. That is, I'd like to be able to do "from zope.interface import ..." and so on.
As I mentioned before, this is already supported: [buildout] parts = ... foo ... [foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy After running the buildout, you will have bin/mypy. If you run bin/mypy, then you'll get a Python prompt and you can import and use anything provided by the listed eggs and their dependencies. You can also do: bin/mypy somescript args and run the script somescript with the given arguments and it will have the path set up for it. One wart is that any scripts defined by the eggs will be installed too when you run the buildout, which you may or may not want. There's a way to suppress this but I should probably add a dedicated interpreter recipe that just creates an interpreter.
Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment
Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout.
(in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them).
Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments. I wouldn't be surprised if Pylons had similar needs. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
Martin Aspeli wrote:
Jim Fulton wrote:
The first step to compatibility is deciding what it means. :) I'm all in favor of workingenv/buildout compatibility. I'd like to see some specifics of how people would like to use workingenv amd buildout together. I have some guesses, but I'd rather hear people say what they want to do. I think this would be much more useful than a discussion of implementation details at this point. Once we know what we want the end result to be, I'm sure you and I can work out some implementation that makes sense. I agree, and I find myself a bit confused by this orientation as well.
The main use case I could imagine wanting to solve would be that I'd like to run 'python' inside zc.buildout and have an interactive prompt that had all the eggs that zc.buildout knew about available. That is, I'd like to be able to do "from zope.interface import ..." and so on.
As I mentioned before, this is already supported:
[buildout] parts = ... foo ...
[foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy
After running the buildout, you will have bin/mypy.
Oh, didn't see that. That's cool! :)
If you run bin/mypy, then you'll get a Python prompt and you can import and use anything provided by the listed eggs and their dependencies.
I guess this wouldn't work for Zope's packages in ${SOFTWARE_HOME}/lib/python because they're not eggs known to buildout, but I could add that to my path manually (and when all of Zope is eggs...)
You can also do:
bin/mypy somescript args
and run the script somescript with the given arguments and it will have the path set up for it.
One wart is that any scripts defined by the eggs will be installed too when you run the buildout, which you may or may not want. There's a way to suppress this but I should probably add a dedicated interpreter recipe that just creates an interpreter.
At least if it was optional to install the eggs' script, that'd be nice. I guess it won't do much harm if the eggs install their scripts, though. Martin
Martin Aspeli wrote:
Jim Fulton wrote:
Martin Aspeli wrote:
Jim Fulton wrote:
The first step to compatibility is deciding what it means. :) I'm all in favor of workingenv/buildout compatibility. I'd like to see some specifics of how people would like to use workingenv amd buildout together. I have some guesses, but I'd rather hear people say what they want to do. I think this would be much more useful than a discussion of implementation details at this point. Once we know what we want the end result to be, I'm sure you and I can work out some implementation that makes sense. I agree, and I find myself a bit confused by this orientation as well.
The main use case I could imagine wanting to solve would be that I'd like to run 'python' inside zc.buildout and have an interactive prompt that had all the eggs that zc.buildout knew about available. That is, I'd like to be able to do "from zope.interface import ..." and so on.
As I mentioned before, this is already supported:
[buildout] parts = ... foo ...
[foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy
After running the buildout, you will have bin/mypy.
Oh, didn't see that. That's cool! :)
If you run bin/mypy, then you'll get a Python prompt and you can import and use anything provided by the listed eggs and their dependencies.
I guess this wouldn't work for Zope's packages in ${SOFTWARE_HOME}/lib/python because they're not eggs known to buildout, but I could add that to my path manually (and when all of Zope is eggs...)
[foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy extra-paths = path-to-your-instance/lib/python
You can also do:
bin/mypy somescript args
and run the script somescript with the given arguments and it will have the path set up for it.
One wart is that any scripts defined by the eggs will be installed too when you run the buildout, which you may or may not want. There's a way to suppress this but I should probably add a dedicated interpreter recipe that just creates an interpreter.
At least if it was optional to install the eggs' script, that'd be nice. I guess it won't do much harm if the eggs install their scripts, though.
It is optional: [foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy extra-paths = path-to-your-instance/lib/python scripts = mypy But more work than I think it should be. The scripts option allows you to select which scripts to include or to rename them. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
[foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy extra-paths = path-to-your-instance/lib/python scripts = mypy
This is great :) I used eggs = ${instance:eggs} to make sure it has the same eggs as our Zope instance, seems to work very well. I don't suppose there's any way to make old-style products (e.g Products.CMFCore) available for import as well? Martin
Martin Aspeli wrote:
Jim Fulton wrote:
[foo] recipe=zc.recipe.egg eggs = egg1 egg2 ... interpreter = mypy extra-paths = path-to-your-instance/lib/python scripts = mypy
This is great :) I used eggs = ${instance:eggs} to make sure it has the same eggs as our Zope instance, seems to work very well.
I don't suppose there's any way to make old-style products (e.g Products.CMFCore) available for import as well?
If they're deployed as eggs (http://wiki.zope.org/zope2/EggifyingZopesExtensionMechanismQuotProductsQuot) then they would be importable. Of course, you'd also need Zope 2 on your PYTHONPATH. -- http://worldcookery.com -- Professional Zope documentation and training Next Zope 3 training at Camp5: http://trizpug.org/boot-camp/camp5
Jim Fulton wrote:
Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment
Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout.
Pylon's doesn't have its own scripts. Generally an application will require Pylons, Pylons requires PasteScript, and PasteScript provides a paster script. A quirk in buildout was that asking for the paster script to go in bin/ didn't seem to work unless I specifically was installing PasteScript. Anyway, they all use entry points, so that part is fine. I've considered encouraging each WSGI/PasteDeploy application to ship its own script that hides the existence of paster (or any specific framework). But that doesn't seem quite right either, since a WSGI application can be (and in production usually is) used as a library in addition to a stand-alone application. At that point it becomes similar to a Zope product, or Plone content type, or... well, I guess there's limits to the similarity as I try to find analogs. I guess that's the style differences between Zope and... all the other frameworks. I'm not sure what name really describes the difference in pattern.
(in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them).
Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments.
I wouldn't be surprised if Pylons had similar needs.
You just have to give the config file as an argument when you start the server, there's nothing setup that binds a script with a config file. I played with a #! setup where the config file became a script, but I found it rather hard to work with (there's a lot of problems with how #! works across platforms). So for now it's just always explicit, and if you want something else you have to set up your own shell scripts. -- Ian Bicking | ianb@colorstudy.com | http://blog.ianbicking.org
On Jan 29, 2007, at 2:25 PM, Ian Bicking wrote:
Jim Fulton wrote:
Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout.
Pylon's doesn't have its own scripts. Generally an application will require Pylons, Pylons requires PasteScript, and PasteScript provides a paster script. A quirk in buildout was that asking for the paster script to go in bin/ didn't seem to work unless I specifically was installing PasteScript. Anyway, they all use entry points, so that part is fine.
Right. The eggs recipe only installs scripts for the named eggs, not for dependencies.
I've considered encouraging each WSGI/PasteDeploy application to ship its own script that hides the existence of paster (or any specific framework). But that doesn't seem quite right either, since a WSGI application can be (and in production usually is) used as a library in addition to a stand-alone application. At that point it becomes similar to a Zope product, or Plone content type, or... well, I guess there's limits to the similarity as I try to find analogs. I guess that's the style differences between Zope and... all the other frameworks. I'm not sure what name really describes the difference in pattern.
I think I do. :) Traditionally, Zope was a pluggable *application*, rather than a library. Typically eggs are meant to be used as libraries. We're moving in the direction of having most of Zope available as eggs that can be used as libraries. Zope 3 is already being widely used that way. In the future there will still be pluggable applications (e.g. Plone) built on libraries provided by the Zope, Plone, and other projects.
(in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them). Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments. I wouldn't be surprised if Pylons had similar needs.
You just have to give the config file as an argument when you start the server, there's nothing setup that binds a script with a config file.
Right. And for convenience, you often want that.
I played with a #! setup where the config file became a script, but I found it rather hard to work with (there's a lot of problems with how #! works across platforms).
yup
So for now it's just always explicit, and if you want something else you have to set up your own shell scripts.
I think that in practice, you'll want to automate this. At least we did. In many cases, you can avoid generating shell scripts and generate Python scripts instead. I like this much better and the techniques that Phillip Eby worked out can be used to make this work nicely on both Unix-based systems and Windows. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Martin Aspeli wrote:
I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation
I would agree that having to mangle multiple scripts is annoying. On the other hand, I find the activate/deactivate dance annoying. I rather think that the scripts that buildout has created should not require any special activation or introduction to a particular environment.
workingenv doesn't require that you activate before running scripts, each script contains in effect what active/deactive does. Basically it munges script creation to do: #!/usr/bin/python -S import sys, os sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'lib/python%s.%s' % (sys.version_info[0], sys.version_info[1])) import site virtual-python -- which might be the better/simpler solution, if you just ignore the tree of symlinks -- selects the environment through the interpreter itself (i.e., #!/path/to/virtual-python/bin/python). Ian
On 25 Jan 2007, at 23:13 , whit wrote:
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
well that's awesome that buildout has duplicated this effort....
It didn't. buildout interfaces with setuptools and easy_install to do its job. The stuff buildout adds is the recipe abstraction so that the installation can be automated over and over again.
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout. is there really a difference between zopectl and source bin/ activate? I guess the main difference here is where PYTHONPATH gets set and how people work with it. for example, if you just start python and want to play with code.... sounds like with zc.buildout you are out of luck for things installed in the buildout.
Possibly, yeah. That's one thing I like about workingenv...
In our situation, we might have multiple versions of software running on different processes started in different workingenv (often not even using zope). being able to contextualize the python path is a useful development tool; this is understandable for buildout to avoid(as a deployment tool), but if we are considering using buildout as a prescribed way for people to manage their development environments, it seems lacking.
You're right, this might just be one of buildout's sore spots, though one that could potentially be fixed. I wonder what it'd take to produce a buildout recipe that added an "activate" script like the one workingenv has... it seems almost trivial.
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv?
I see no problem with that. zc.buildout is one way of deploying Python software like Zope. As long as you've got eggs, you could install them manually into your workingenv just fine. buildout just does it an automated fashion (and, of course, it can do more than just installing eggs).
which I could automate in my env (workingenv takes recipes also). for that matter, eggs automate the installation of eggs...
Yup, you're right. Note that buildout does more than just installing eggs, though. It can automate potentially anything). I find the zc.recipe.cmmi recipe quite powerful, it can build any Unixy software (e.g. Python, libxml, ...) using "configure; make; make install".
I'm not sure what it would take to make buildout just automate install eggs into my env... but not putting them in special directories would be a start.
As said, I agree.
currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes.
for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install.
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is. +1
this may be all that is required to make the two play well together (or have buildout respect an existing workingenv when doing it's local install of eggs). Maybe it's just a matter of zc.buildout seeing workingenv is in effect and not composing special pythonpaths.
harmony is my interest here. workingenv is pretty low-level and works hard to work well with python. there is no reason that zc.buildout should have a problem with that.. it just needs to do less when appropriate.
Well, it seems they already can work together, given Hanno's implementation for a workingenv recipe. The question is whether it makes sense. If you're primiarly interested in the automated installation of eggs, it seems workingenv is just another unnecessary layer (or buidout, whichever way you see it).
as stated before, I don't mind using zc.buildout, but I don't want to have to learn zc.buildout to use it meaningfully in my existing setup. If a buildout recipes could be executed by themselves(like buildout-zope2, buildout-deliverance, buildout- squid) as well as by aggregated recipes. This would make buildout a killer tool inside my workingenv rather than a choice I had to make between two technologies.
As said already, I think once you've got buildout, there's no need for workingen, except if you think that "Zope stinks" ;)
ooooooh philip...
<rant> you're drawing my ire for taking a stereotypical stone-age isolationist zope developer attitude toward the rest of the world. shame on you(though I don't think you really mean this. so take this as feedback).
No I didn't mean it. Perhaps adding that little tongue-in-cheek there was a mistake, I certainly didn't want to cause a rant there... I spoke from a technical point of view, based on the fact that workingenv and buildout have a lot of common features. Using both at the same time seems like double trouble to me (adding unnecessary layers, etc.). Perhaps it isn't, perhaps it makes total sense (you certainly seem to have a point with the Python interpreter). All I wanted to say is that if you guys get good mileage out of workingenv, I think you should keep using it. Why couldn't Plone be deployable in either a workingenv setup or a buildout? I don't think we ever have to make an exclusive decision for either one. As for kool aids and smelly things, the general reaction I get from people I show workingenv to is mixed as well (I quite like it myself and am more familiar with it than buildout). Not everyone likes the bin/activiate script and the prompt maniuplation either. Yes, workingenv might be lighter on the kool-aid, but it's not like zc.buildout is a big pile of magic, either. And in particular, I don't think zc.buildout tears down a lot of bridges. In fact, a lot of the things I see in it seem like they were done the way they are done in order to use existing bridges (eggs, Unixy applications, heck even using ConfigParser instead of ZConfig...) As a pure *consumer* of both workingenv and zc.buildout, I haven't yet reached the zen that I'd like to have to suggest the best way toward harmony, but I would love to discuss ideas at the Camp5 sprint. Philipp
I hate to jump into this thread .... but I'll make a few comments. On Jan 25, 2007, at 5:13 PM, whit wrote:
Philipp von Weitershausen wrote:
whit wrote:
Martin Aspeli wrote:
Philipp von Weitershausen wrote:
This is awesome, and by that I don't mean the fact that we have a plone buildout, but that we actually have Zope 2 recipes for buildout. I hope they can be moved to svn.zope.org for further development to benefit the whole Zope 2 community.
I believe this is just a matter of contrib agreements being sorted out (Hanno?). I guess I need to get mine sorted out as well if I'm going to keep working on this when it moves... :-/
I also see that workingenv was abandoned. That's very good to hear because buildout has a lot of machinery for installing eggs already, it would just've been duplicated with workingenv...
is there some advantage to the way that buildout handles eggs over workingenv. as I understand it, workingenv *only* handles python setup and does that well and transparently.
Right, buildout handles a lot more. For example, we use buildout to build C libraries and to generate various sorts of application artifacts like configuration files, custom startup scripts, and so on, in addition to Python scripts.
The point is that buildout *already* handles eggs. There's really no point for having an extra layer on top of buildout. The zc.recipe.egg recipe can install any egg (as a development one or not) in an automated fashion, which is exactly what you'd want from a buildout.
well that's awesome that buildout has duplicated this effort....
I'm not sure what effort you think buildout is duplicating. WRT eggs, buildout, like workinenv/easy_install builds on setuptools. setuptools does the heavy lifting with regard to building eggs. Buildout does have some nice features for building eggs with extensions that need custom setup options. Like workenv, buildout supports installing eggs in development environments rather than the system Python. workingenv bends easy_install to it's will, while buildout goes below easy_install. This allows buildout to have more control, which was important to me. Buildout gives me greater control over egg revisions used and the way that scripts get generated. (Maybe workingenv would give me more control than I think it does -- I haven't looked at it in a while.)
the "source bin/activate" dance is the only thing I see being a detriment here(and with the latest workingenv, your shell prompt lets you know you are in an env).
Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout. is there really a difference between zopectl and source bin/ activate? I guess the main difference here is where PYTHONPATH gets set and how people work with it. for example, if you just start python and want to play with code.... sounds like with zc.buildout you are out of luck for things installed in the buildout.
I'm not sure what you mean here. I'll note that buildout lets you easily create "interpreter" scripts. When run without arguments, you get a Python prompt. When run with a script name and arguments, then they will run the script with the arguments. You define an interpreter by specifying on or more eggs and those eggs and their dependencies (and additional paths you can optionally set) will be included in the interpreter's path. This makes playing with code pretty easy.
In our situation, we might have multiple versions of software running on different processes started in different workingenv (often not even using zope). being able to contextualize the python path is a useful development tool; this is understandable for buildout to avoid(as a deployment tool), but if we are considering using buildout as a prescribed way for people to manage their development environments, it seems lacking.
I don't know what you mean by "contextualize the python path". An important feature of buildout (actualy the egg recipe) is that it lets you create scripts with exactly the eggs they need. This was a core motivation of buildout. And, as I have mentioned above, you can also create interpreter scripts for experimentation and running other scripts. Perhaps these interpreter scripts are like your workingenvs.
Workingenv made it more complex than it needed to be (or buildout did, depending on which perspective you look at it from). I believe Hanno wanted to rescue the recipe in case others found it useful, but it's not used for now.
what about if I'm already using workingenv... and want to use zope or plone in my workingenv?
I see no problem with that. zc.buildout is one way of deploying Python software like Zope. As long as you've got eggs, you could install them manually into your workingenv just fine. buildout just does it an automated fashion (and, of course, it can do more than just installing eggs).
which I could automate in my env (workingenv takes recipes also). for that matter, eggs automate the installation of eggs...
Actually, they don't. They allow automation and workingenv and buildout provide automation, both building on setuptools.
I'm not sure what it would take to make buildout just automate install eggs into my env...
I'm not clear on what the advantage would be. I'm probably missing some use cases. I think they are both valid approaches to the problem.
but not putting them in special directories would be a start.
Um, both workingenv and buildout install eggs into specific/special directories. You can configure buildout to install eggs wherever you want, including into the directory structure that workingenv uses. Note that buildout separates regular and develop eggs to allow regular eggs to be shared across projects. This can be a big win for performance, space and allowing buildouts to be created offline.
currently, I don't see an easy way to use buildouts inside a workingenv, whereas the rest of python world works great. I will have alot of trouble explaining to my developer who already think zope smells that they have to change the way they work to use zc.buildout recipes.
for example, I can't use the deliverance or lxml buildout with an existing topp.deploy workingenv because of buildout's arbitrary egg handling scheme. If zc.buildout didn't try to do so much, the python would be installed transparently like everything else I easy_install.
I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is.
You can set the eggs directory to anything you want: [buildout] eggs-directory = lib/python develop-eggs-directory = lib/python If you want, you could put this in ~/.buildout/default.cfg and buildout will work this way for all of your buildouts.
this may be all that is required to make the two play well together (or have buildout respect an existing workingenv when doing it's local install of eggs). Maybe it's just a matter of zc.buildout seeing workingenv is in effect and not composing special pythonpaths.
harmony is my interest here. workingenv is pretty low-level and works hard to work well with python. there is no reason that zc.buildout should have a problem with that.. it just needs to do less when appropriate.
If people really see value in having buildout and workingenv work together, perhaps Ian and I could explore this a bit at PyCon next month.
try to justify why a zope hammer is better and why zope folk keep building new hammers that don't play nice with everyone else's hammer loses us cred.
You seem to be implying that workingenv is somehow a standard tool and that buildout is a zope-specific tool that needlessly duplicates a standard tool. zc.buildout is in no way zope specific. Can a Zope developer not develop a tool without it being stamped as "zope specific"? zc.buildout and workingenv started life at about the same time. They are both valid and useful tools with somewhat different goals and approaches. They both deserve consideration. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
participants (8)
-
Alexander Limi -
Ian Bicking -
Jim Fulton -
Martin Aspeli -
Philipp von Weitershausen -
Rob Miller -
Tres Seaver -
whit