[Zope-dev] ploneout - Or how using zc.buildout for a common Zope2 project might look like

[Hanno Schlichting]

Hi.

(I already send this mail through gmane but it didn't show up on the
lists for half a day. So it might show up twice later on, sorry.)

Spawning of a discussion on plone-devel
(http://thread.gmane.org/gmane.comp.web.zope.plone.devel/13865/focus=13865)
about how to use eggs and their features like entrypoints in our current
development environments, we discussed some of the available options. As
I think this is a common problem for all Zope2 developers I'll post this
to zope-devel as well.

In order to get a better understanding of how one of the possible
solutions called zc.buildout works, I started to write all those recipes
one would need to set up a full development environment for Plone 3.0.
Quite frankly most of the work was already done by Jim and I only needed
to adjust some of his Zope3 specific recipes  ;)

A full Plone 3.0 environment includes (should be similar for other
projects):

- Checking out the Zope 2.10 branch and compiling the C extensions
- Creating a Zope instance
- Checking out one or many SVN bundles containing a multitude of
Products and moving these into the instance
- Checking out a product from CVS and moving it into the instance
(hopefully you don't need to do that anymore  :)
- Installing some external Python packages (elementtree, PIL, py-openid,
...) and making those available in the Zope instance
- Somehow putting 19 Python packages in development mode into the
instance (these are the ones we have written ourselves)

I have put those recipes that where necessary to automate all of these
steps into the ploneout package
(https://svn.plone.org/svn/plone/ploneout/trunk) but intend to move them
to svn.zope.org as soon as I have commit rights and should people be
interested in them.

The four recipes that I have written, are able to check-out Zope2 and
compile it inplace, create a Zope2 instance, check out a bundle (any
folder really) from either SVN or CVS and finally enable a folder as a
working environment (http://www.python.org/pypi/workingenv.py).

Among the more useful features are:

- All recipes support both the global as well as a part specific offline
option, so you can tell each checkout separately, that it should not do
anything by itself after the first checkout. This can cut down the time
needed to update the buildout quite enormously.

- The Zope2 instance recipe takes a product-set option. You can give it
a list of bundles and it will symlink all folders included in each
bundle into the instance.

- The Zope2 instance recipe also supports the zcml option known from
it's Zope3 counterpart, so you can create ZCML slugs automatically.

- The workingenv recipe takes any folder (usually a Zope2 instance) and
turns it into a workingenv, supporting all the command line options
workingenv itself support. Among these is also the requirements option
that lets you specify a text file (url or local file) which includes
dependencies to be installed into the workingenv.

- And as another special option the workingenv recipe supports an
additional develop-set option which like the Zope instance product-set
option takes a bundle (any local folder really) and calls setup.py
develop on any package it finds in there. Usually this installs all
those packages into INSTANCE_HOME/lib/python (the workingenv) in
development mode.


If you combine all of those recipes (like it's done in the
ploneout/buildout.cfg) you get a working Zope2 instance and have various
ways of installing both products and packages into it.

Currently you won't get a running Plone 3.0 instance as not all of the
required newly written Python packages are setuptools enabled yet.

What's missing now is a more seamless integration of the packages or
eggs installed into the workingenv and those installed in the global
buildout itself. Secondly in order to be productive one would need some
more nice scripts in the global buildout/bin that would allow to start
the Zope instance, run tests from any development egg, Product or Zope
itself reliably, a recipe to automatically populate the ZODB with some
initial content like a Plone site... and the usual Windows support that
isn't yet there but could be easily added if people show any interest in
those recipes.

Currently this was only an exercise to support development environments,
for deployment you would want to use released versions which would need
some new recipes.

As this is my first attempt to use zc.buildout I might have made some
obvious mistakes not recognizing it's potential or working against its
design. Please tell me about those mistakes  :)

My current conclusion for using zc.buildout is that it makes setting up
a complete environment from scratch a lot easier and should help
developers that are not that advanced yet or are not developers at all.
Think of the first day at a sprint for example, where you spent time
helping people to get a running system or doing that with your designers.

For advanced developers it constrains them in a way they might not find
acceptable - we all have our different ways of managing our projects
right now. If you use buildout for a complex project you can either use
it or not, you cannot really use only parts of it.

Comments and suggestions are welcome as always  ;)

Hanno