[Zope-dev] env var support for zc.buildout

[Florian Friesdorf]

On Fri, Apr 09, 2010 at 10:16:51AM -0400, Jim Fulton wrote:
> > We currently use it for 5 very similar sites that share one repository
> > but use 5 checkouts of it:
> >
> > base.cfg
> > [buildout]
> > parts = instance
> >
> > develop = ...
> > eggs = ...
> > zcml = ...
> >
> > [instance]
> > ...
> >
> >
> > site-1.cfg
> > [buildout]
> > eggs +=
> > zcml +=
> >
> >
> > testenv.cfg
> > [buildout]
> > extends =
> >    base.cfg
> >    ${env:site}.cfg
> >
> > <stuff for testenv>
> >
> >
> > deploy.cfg
> > [buildout]
> > extends =
> >    base.cfg
> >    ${env:site}.cfg
> >
> > <stuff for deploy>
> >
> >
> > % site=site-1 ./bin/buildout -c deploy.cfg
> 
> Your extends usage seems a bit convoluted.  Why not just have:
> 
> - deploy.cfg extend base.cfg
> 
> - site1-1.cfg etc extend deploy.cfg
> 
> Then you'd just:
> 
> % ./bin/buildout -c site-1.cfg

That would mean:
- 1 base.cfg
- 1 deploy.cfg
- 1 testenv.cfg
- 5 site-specific extending deploy.cfg
- 5 site-specific extending testenv.cfg
total: 13 files

Site-specific information reside in two cfgs, which is error-prone.


In contrast to our current setup, with env var support, without
the need to duplicate information.
- 1 base.cfg
- 1 deploy.cfg
- 1 testenv.cfg
- 5 site specific cfgs
total: 8 files


Site-specific information could be factored out, as in the setup with
env var support:

testenv-site-1.cfg:
[buildout]
extends =
    testenv.cfg
    site-1.cfg

Information would not be duplicated but the amount of files increases to
18.


Further we use a dev environment tailored for local individual
development in contrast to the testenv which is meant for the customer
to be kept up-to-date about the development, without e.g. PDBDebugMode.
With that dev environment the numbers are 18 (with site-specific info
in three places) or 23 vs. 9 files.

This idea can be taken further to mass hosting in dedicated instances
but based on one buildout, or branches of one buildout that share the
majority of stuff.

Other use cases coming to my mind:
- ip address for zope to listen on: instead of hard-coded 8080 or
  127.0.0.1:8080, you can have ${env:MYPROJECT_LISTEN} and people being
  involved in several projects can choose in order to have several
  buildouts running in parallel
- location of the Data.fs: I like to have the Data.fs outside of the
  buildout dir for development, which enables the use of 'git clean
  -xdf', i.e. wiping everything which is not tracked.


I like the idea of the need to configure the name of the environment
section:
[buildout]
environment=env
extends = ${env:...}

At the time the extends are processed, the whole buildout section is
already read into the options dict, so configuration of the name of the
environment section is easily possible and can be used to trigger the
substitution code:
if extends:
   envsecname = options.get('environment')
   if envsecname:
   	# do variable substitution like in my patch
	# if no environment is specified nothing changes
   # continue normally

I would not pop() the options, so the name of the environment section
can be queried by other parts of buildout. Likewise, extends could not
be pop()'ed - both then would need to go onto a list, which buildout
checks, in order not to complain about them being unused.


If I could get commit acces to svn.zope.org, I would further work on
this in a separate branch, more effective than via patches. Committer
agreement I can fax/email. I am not committing to branches that don't
belong to me, without prior consultation. So far I have commit access to
the plone repo, which might be meaningless or at least a minor sign of
credibility.


I hope I could clarify the usefulness of env var support.

florian

-- 
Florian Friesdorf <flo at chaoflow.net>
  GPG FPR: EA5C F2B4 FBBB BA65 3DCD  E8ED 82A1 6522 4A1F 4367
Jabber/XMPP: flo at chaoflow.net
  OTR FPR: 9E191746 213321FE C896B37D 24B118C0 31785700
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20100409/9a8c386e/attachment.bin