[Zope-dev] Best way to invoke paster/WSGI apps on cmdline

Sat Apr 18 18:10:26 EDT 2009

Hi there,

Christian Theune wrote:

> On Thu, 2009-04-16 at 01:01 +0200, Hanno Schlichting wrote:
> > Hi.

Sorry Hanno, your posting got lost on the way to my mailreader. I
noticed you replied when reading Theuni's posting.

> > Uli Fouquet wrote:
> > > As Grok is now approaching the 1.0 release we (i.e. some Grok developers
> > > and me) want to have a 'compliant' way to offer paster-based serving of
> > > Grok instances.
> > 
> > I'd love to have the same for Plone, but failed to find any canonical
> > definition of "compliant" the same way you did.

Glad to hear that others have the same problems :)

> > > Here's a short (probably incomplete) list of some 'full-stack'
> > > paster-capable 'frameworks' and how they invoke paster from the
> > > commandline (please correct/extend if I am wrong/incomplete here):
> > > 
> > >  ================ =================== ============================
> > >     Framework      Ini-file location        Paster call         
> > >  ================ =================== ============================
> > >  grokproject      parts/etc/          bin/paster serve \
> > >                                          parts/etc/deploy.ini
> > >  repoze.bfg       <project root>      paster serve MyProject.ini
> > >  turbogears2      <project root>      paster serve development.ini
> > >  pylons           <project root>      paster serve development.ini
> > >  zopeproject      etc/                bin/paster serve \
> > >                                          etc/deploy.ini
> > >  z3c.r.paster     parts/<app>/        bin/app
> > 
> > I'll add the current Plone 4 way:
> > 
> > Plone      parts/instance/etc      bin/instance-fg
> > 
> > where instance-fg is a wrapper that calls:
> > 
> > bin/paster serve parts/instance/etc/paste.ini

We thought about this as well.

> Some motivators for me, as I saw some of those variants around and some
> input with my experience from the buildout business:
> 
> - As a Unix-Lover, I do appreciate starting services having a SYSV style
>   call, which means: (somecommand) start/stop/restart/...
> 
>   Thus no calls that require me to remember a config file location and
>   pass it on.
> 
>   This also means that (somecommand) needs to be configurable as I might
>   want to deploy multiple buildouts into the same machine having the
>   deployment options create the service scripts in /etc/init.d and thus
>   they can't all be called "instance". Having it named "myproject" seems
>   better to me.

I see. I am not sure, everybody has this requirement, but we should keep
it in mind. The question here is: how do you name your commands then? Do
you opt for (somecommand) --debug or similar in that case?

> - Also, I do like my autocomplete to leave me with full command name
>   completions, thus: (somecommand)-ctl
>   (somecommand)-debug doesn't work well for me: I always have to start
>   typing again to complete the command *and* give a parameter.
>   (Remember  that I want parameters so I have SYSV compatible init
>   commands right away)
> 
> - In a buildout environment, the use of .in files breaks the flexibility
>   that the various buildout mechanism deliver to us: extends, variable
>   inclusion, etc. have to be built again and again and again in the
>   recipes. We started out with zope3instance recipes that did that but
>   finally moved away from this.

Interesting. This, at least with Grok, can become a serious problem. We
currently have five different configuration files generated by
`grokproject`. Admitted, that one might skip one (`debug.ini`), we are
left with four, which require three different configuration grammars:

 zope.conf, zdaemon.conf: ZConf grammar
 deploy.ini: ConfigParser grammar
 site.zcml: ZCML

but this is a different topic.

Some of those files (namely the .ini files) meanwhile have grown large
defining loggers and all that. Putting them into buildout.cfg makes it
nearly unreadable. It becomes even difficult to check, which part of the
configuration you're currently watching. People with more advanced
setups might run into even greater trouble.

So, the pros and cons of external vs. buildout.cfg-embedded config files
might read like this (for embedded configs):

* pros:

  - supports all buildout mechanisms 
    (namely: var interpolation and extend)

  - all configs in one file (you know where to look for them)

* cons:

  - harder to read

  - confusing syntax (mix of at least three grammars)

  - impossible to split permissions on parts of the config

With external configs it's the opposite picture. Can you tell more about
the problems with zope3instance?

> - Do not make me check in generated things.

Of course. As with buildout we _will_ have generated files (opposite to
most other frameworks). This means that we cannot put flat ini files in
the project/instance root (as they won't be flat files but generated
ones).

> - If you'd like people that know things like paster to pick up our
>   environment easily, I think, that: make it obvious where the
>   deploy.ini & Co went and as they are generated files, leave a visible
>   note at the beginning of the file, that states something like:
> 
>   # NOTE: This file is generated by foo.recipe.paster, it will be
>   # overwritten. Please apply changes to your buildout configuration
>   # (located in XXX) instead of changing this file.

Yep. Thanks for all the input!

[snip: zopeproject and referencing 'instances']

If I try to conclude, then I see several decisions to make:

* "paster serve ..." vs. "commandname" approach.

  It looks like that the "commandname" approach makes more sense with
  Zope as we certainly want to use the power of buildout and therefore
  have to use generated config files in 'exotic' (seen from the casual
  paster user's point of view) places.

* "commandname-dbg" vs. "commandname --debug"

  This means, we have to decide, whether we want several commands
  performing special tasks (most probably: running everything in a 
  special 'mode'), or one command that accepts options to indicate
  that special behaviour expected.

* "conffile.in" vs. buildout.cfg-embedded config files

  I personally would really prefer the first approach for ease of use.
  If, however, this leads to problems with parsing the files, we are
  more or less bound to the latter. I'd like to investigate that
  further.

It looks like that the 'commandname-w/o-configfile-param' approach suits
buildout-based environments best, which might be a good reason to be
different in that respect from other paster-frameworks.

From my initial list of invokes (enriched by Hanno's Plone4 approach)
then only some remain:

- "instance-fg <options>" and friends

- "mycommand [start|stop|status...] <options>"

The latter is partly already implemented by z3c.recipe.paster. Currently
it does not support external config files and might do a bit more than 
needed (for example it calls ZConfig parser in schemaless mode, but
there certainly is a good reason for it; in grokprojects we let
zope.app.wsgi do this stuff), but it took me less than half an hour to
create a flatter version that accepts both: internal and external config
files (where external zope.confs are parsed with schemata) so that one
can start a paster served grokproject with generated configuration files
in parts/etc by doing::

 $ bin/myapp

in foreground, where the script name is built from the buildout.cfg
section name. It is actually a wrapper that runs::

  paster serve buildout-set-ini-file [cmdline-options]

This means also, you have to do::

  $ bin/myapp --daemon

to start the server in background (which is bad if you prefer SYSV
compatible behaviour, i.e. ``bin/myapp start`` here), but you can do::

  $ bin/myapp status

  $ bin/myapp stop

which does, what one expects.

One can also pass the usual paster options like ``--reload``. There is,
however, no possibility to start in debug mode, i.e. with a debug.ini as
argument, except you declare another command (at least I don't see any
way to do it different).

It should also be possible to make use of z.r.paster in Plone4, as one
can easily generate an ``instance-fg`` script this way. I wonder,
however, whether this is still wanted with paster.

Hm, all this is confusing (and much too long). Maybe someone
nevertheless has an opinion?

Best regards,

-- 
Uli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20090419/4f90dcfd/attachment.bin 