Best way to invoke paster/WSGI apps on cmdline
Hi there, As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances. This means mainly that when a user creates a project with `grokproject`, we want to have the config files (deploy.ini, zope.conf, etc.) generated in a location, that users expect. Some 'standard' location if such a thing should exist. This also applies to the name/functionality of the script to start the paster server. Currently we do it like this with `grokproject`: Configuration files (deploy.ini, debug.ini, zope.conf, site.zcml, zdaemon.conf) are generated by zc.buildout in a local parts/etc/ directory from templates in a project-local etc/ directory. Furthermore we let buildout create a script ``bin/paster``, so that you can start your instance with:: $ bin/paster serve parts/etc/deploy.ini Nobody seems to like this (including myself, although I am mainly to blame for that). Especially the parts/etc/deploy.ini path is too 'hidden'. Time to learn from others. Other frameworks (repoze.bfg, TurboGears for example) do it differently and it looks like there is also the z3c.recipe.paster-way to do it, as Roger Ineichen stated yesterday on this list. Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here): ================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app So we have two things to decide: (1) how do we want to let people invoke paster and (2) where do we generate the ini-files? I guess there are pros and cons for every of the approaches and we already discussed some of them in the Grok community, but the most important thing is, that we would like to keep Grok compatible with other frameworks/approaches, so that people feel a bit more used to it when they try it for the first time. Collected (incomplete) cons: - grokproject: hidden ini-file path, complicate paster invoke - zopeproject: supports only relative paths, one cannot invoke /abs/path/to/project/bin/paster serve ... from outside the project dir - z3c.recipe.paster: not sure whether commandline args like ``--reload`` are respected; invocation is different from other paster-based frameworks (this approach, however, looks most attractive to me). - turbogears2/pylons: not buildout based by default; makes it harder to create collaborative SVN-based projects. At least it would be nice to have a common approach or 'best practice' recommendation when it comes to Zope based paste deploys. It would be even nicer to have an approach that is similar also to non-Zope environments. We'd like to follow such a convention with `grokproject` and learn from other's expericences in that matter. Best regards, -- Uli
Hi. Uli Fouquet wrote:
As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances.
I'd love to have the same for Plone, but failed to find any canonical definition of "compliant" the same way you did.
Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here):
================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app
I'll add the current Plone 4 way: Plone parts/instance/etc bin/instance-fg where instance-fg is a wrapper that calls: bin/paster serve parts/instance/etc/paste.ini Another option I considered was the way zopeproject does it. Except that I still have to give a path to an "instance home" in a zope.conf file, which either needs to be filled in by buildout or otherwise '..' needs to be handled correctly for the etc/ case.
At least it would be nice to have a common approach or 'best practice' recommendation when it comes to Zope based paste deploys. It would be even nicer to have an approach that is similar also to non-Zope environments. We'd like to follow such a convention with `grokproject` and learn from other's expericences in that matter.
My conclusion on this so far is: it's a matter of taste for the most part and there doesn't seem to be any good convincing argument for any of the variants. The way I choose it for Plone 4 right now is basically to minimize the difference to the former invocations. The main reason for the former state of affairs is however to support multiple ZEO clients inside the same buildout. I'm not sure if this use-case exists for Paste or if multiple processes are usually handled by using some form of mod_wsgi anyways. Hanno
Hi, On Thu, 2009-04-16 at 01:01 +0200, Hanno Schlichting wrote:
Hi.
Uli Fouquet wrote:
As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances.
I'd love to have the same for Plone, but failed to find any canonical definition of "compliant" the same way you did.
Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here):
================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app
I'll add the current Plone 4 way:
Plone parts/instance/etc bin/instance-fg
where instance-fg is a wrapper that calls:
bin/paster serve parts/instance/etc/paste.ini
Some motivators for me, as I saw some of those variants around and some input with my experience from the buildout business: - As a Unix-Lover, I do appreciate starting services having a SYSV style call, which means: (somecommand) start/stop/restart/... Thus no calls that require me to remember a config file location and pass it on. This also means that (somecommand) needs to be configurable as I might want to deploy multiple buildouts into the same machine having the deployment options create the service scripts in /etc/init.d and thus they can't all be called "instance". Having it named "myproject" seems better to me. - Also, I do like my autocomplete to leave me with full command name completions, thus: (somecommand)-ctl (somecommand)-debug doesn't work well for me: I always have to start typing again to complete the command *and* give a parameter. (Remember that I want parameters so I have SYSV compatible init commands right away) - In a buildout environment, the use of .in files breaks the flexibility that the various buildout mechanism deliver to us: extends, variable inclusion, etc. have to be built again and again and again in the recipes. We started out with zope3instance recipes that did that but finally moved away from this. - Do not make me check in generated things. - If you'd like people that know things like paster to pick up our environment easily, I think, that: make it obvious where the deploy.ini & Co went and as they are generated files, leave a visible note at the beginning of the file, that states something like: # NOTE: This file is generated by foo.recipe.paster, it will be # overwritten. Please apply changes to your buildout configuration # (located in XXX) instead of changing this file.
Another option I considered was the way zopeproject does it. Except that I still have to give a path to an "instance home" in a zope.conf file, which either needs to be filled in by buildout or otherwise '..' needs to be handled correctly for the etc/ case.
If your buildout does the right thing, then you should be able to abstract that complication away leaving only - a reference to a configuration file - maybe a working directory change Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development
Hi there, Christian Theune wrote:
On Thu, 2009-04-16 at 01:01 +0200, Hanno Schlichting wrote:
Hi.
Sorry Hanno, your posting got lost on the way to my mailreader. I noticed you replied when reading Theuni's posting.
Uli Fouquet wrote:
As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances.
I'd love to have the same for Plone, but failed to find any canonical definition of "compliant" the same way you did.
Glad to hear that others have the same problems :)
Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here):
================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app
I'll add the current Plone 4 way:
Plone parts/instance/etc bin/instance-fg
where instance-fg is a wrapper that calls:
bin/paster serve parts/instance/etc/paste.ini
We thought about this as well.
Some motivators for me, as I saw some of those variants around and some input with my experience from the buildout business:
- As a Unix-Lover, I do appreciate starting services having a SYSV style call, which means: (somecommand) start/stop/restart/...
Thus no calls that require me to remember a config file location and pass it on.
This also means that (somecommand) needs to be configurable as I might want to deploy multiple buildouts into the same machine having the deployment options create the service scripts in /etc/init.d and thus they can't all be called "instance". Having it named "myproject" seems better to me.
I see. I am not sure, everybody has this requirement, but we should keep it in mind. The question here is: how do you name your commands then? Do you opt for (somecommand) --debug or similar in that case?
- Also, I do like my autocomplete to leave me with full command name completions, thus: (somecommand)-ctl (somecommand)-debug doesn't work well for me: I always have to start typing again to complete the command *and* give a parameter. (Remember that I want parameters so I have SYSV compatible init commands right away)
- In a buildout environment, the use of .in files breaks the flexibility that the various buildout mechanism deliver to us: extends, variable inclusion, etc. have to be built again and again and again in the recipes. We started out with zope3instance recipes that did that but finally moved away from this.
Interesting. This, at least with Grok, can become a serious problem. We currently have five different configuration files generated by `grokproject`. Admitted, that one might skip one (`debug.ini`), we are left with four, which require three different configuration grammars: zope.conf, zdaemon.conf: ZConf grammar deploy.ini: ConfigParser grammar site.zcml: ZCML but this is a different topic. Some of those files (namely the .ini files) meanwhile have grown large defining loggers and all that. Putting them into buildout.cfg makes it nearly unreadable. It becomes even difficult to check, which part of the configuration you're currently watching. People with more advanced setups might run into even greater trouble. So, the pros and cons of external vs. buildout.cfg-embedded config files might read like this (for embedded configs): * pros: - supports all buildout mechanisms (namely: var interpolation and extend) - all configs in one file (you know where to look for them) * cons: - harder to read - confusing syntax (mix of at least three grammars) - impossible to split permissions on parts of the config With external configs it's the opposite picture. Can you tell more about the problems with zope3instance?
- Do not make me check in generated things.
Of course. As with buildout we _will_ have generated files (opposite to most other frameworks). This means that we cannot put flat ini files in the project/instance root (as they won't be flat files but generated ones).
- If you'd like people that know things like paster to pick up our environment easily, I think, that: make it obvious where the deploy.ini & Co went and as they are generated files, leave a visible note at the beginning of the file, that states something like:
# NOTE: This file is generated by foo.recipe.paster, it will be # overwritten. Please apply changes to your buildout configuration # (located in XXX) instead of changing this file.
Yep. Thanks for all the input! [snip: zopeproject and referencing 'instances'] If I try to conclude, then I see several decisions to make: * "paster serve ..." vs. "commandname" approach. It looks like that the "commandname" approach makes more sense with Zope as we certainly want to use the power of buildout and therefore have to use generated config files in 'exotic' (seen from the casual paster user's point of view) places. * "commandname-dbg" vs. "commandname --debug" This means, we have to decide, whether we want several commands performing special tasks (most probably: running everything in a special 'mode'), or one command that accepts options to indicate that special behaviour expected. * "conffile.in" vs. buildout.cfg-embedded config files I personally would really prefer the first approach for ease of use. If, however, this leads to problems with parsing the files, we are more or less bound to the latter. I'd like to investigate that further. It looks like that the 'commandname-w/o-configfile-param' approach suits buildout-based environments best, which might be a good reason to be different in that respect from other paster-frameworks. From my initial list of invokes (enriched by Hanno's Plone4 approach) then only some remain: - "instance-fg <options>" and friends - "mycommand [start|stop|status...] <options>" The latter is partly already implemented by z3c.recipe.paster. Currently it does not support external config files and might do a bit more than needed (for example it calls ZConfig parser in schemaless mode, but there certainly is a good reason for it; in grokprojects we let zope.app.wsgi do this stuff), but it took me less than half an hour to create a flatter version that accepts both: internal and external config files (where external zope.confs are parsed with schemata) so that one can start a paster served grokproject with generated configuration files in parts/etc by doing:: $ bin/myapp in foreground, where the script name is built from the buildout.cfg section name. It is actually a wrapper that runs:: paster serve buildout-set-ini-file [cmdline-options] This means also, you have to do:: $ bin/myapp --daemon to start the server in background (which is bad if you prefer SYSV compatible behaviour, i.e. ``bin/myapp start`` here), but you can do:: $ bin/myapp status $ bin/myapp stop which does, what one expects. One can also pass the usual paster options like ``--reload``. There is, however, no possibility to start in debug mode, i.e. with a debug.ini as argument, except you declare another command (at least I don't see any way to do it different). It should also be possible to make use of z.r.paster in Plone4, as one can easily generate an ``instance-fg`` script this way. I wonder, however, whether this is still wanted with paster. Hm, all this is confusing (and much too long). Maybe someone nevertheless has an opinion? Best regards, -- Uli
Hi all
Betreff: Re: [Zope-dev] Best way to invoke paster/WSGI apps on cmdline
Hi there,
Christian Theune wrote:
On Thu, 2009-04-16 at 01:01 +0200, Hanno Schlichting wrote:
Hi.
Sorry Hanno, your posting got lost on the way to my mailreader. I noticed you replied when reading Theuni's posting.
Uli Fouquet wrote:
As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances.
I'd love to have the same for Plone, but failed to find any canonical definition of "compliant" the same way you did.
Glad to hear that others have the same problems :)
Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here):
================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app
I'll add the current Plone 4 way:
Plone parts/instance/etc bin/instance-fg
where instance-fg is a wrapper that calls:
bin/paster serve parts/instance/etc/paste.ini
We thought about this as well.
Some motivators for me, as I saw some of those variants around and some input with my experience from the buildout business:
- As a Unix-Lover, I do appreciate starting services having a SYSV style call, which means: (somecommand) start/stop/restart/...
Thus no calls that require me to remember a config file location and pass it on.
This also means that (somecommand) needs to be configurable as I might want to deploy multiple buildouts into the same machine having the deployment options create the service scripts in /etc/init.d and thus they can't all be called "instance". Having it named "myproject" seems better to me.
I see. I am not sure, everybody has this requirement, but we should keep it in mind. The question here is: how do you name your commands then? Do you opt for (somecommand) --debug or similar in that case?
- Also, I do like my autocomplete to leave me with full command name completions, thus: (somecommand)-ctl (somecommand)-debug doesn't work well for me: I always have to start typing again to complete the command *and* give a parameter. (Remember that I want parameters so I have SYSV compatible init commands right away)
- In a buildout environment, the use of .in files breaks the flexibility that the various buildout mechanism deliver to us: extends, variable inclusion, etc. have to be built again and again and again in the recipes. We started out with zope3instance recipes that did that but finally moved away from this.
Interesting. This, at least with Grok, can become a serious problem. We currently have five different configuration files generated by `grokproject`. Admitted, that one might skip one (`debug.ini`), we are left with four, which require three different configuration grammars:
zope.conf, zdaemon.conf: ZConf grammar deploy.ini: ConfigParser grammar site.zcml: ZCML
but this is a different topic.
Some of those files (namely the .ini files) meanwhile have grown large defining loggers and all that. Putting them into buildout.cfg makes it nearly unreadable. It becomes even difficult to check, which part of the configuration you're currently watching. People with more advanced setups might run into even greater trouble.
So, the pros and cons of external vs. buildout.cfg-embedded config files might read like this (for embedded configs):
* pros:
- supports all buildout mechanisms (namely: var interpolation and extend)
- all configs in one file (you know where to look for them)
* cons:
- harder to read
- confusing syntax (mix of at least three grammars)
- impossible to split permissions on parts of the config
With external configs it's the opposite picture. Can you tell more about the problems with zope3instance?
- Do not make me check in generated things.
Of course. As with buildout we _will_ have generated files (opposite to most other frameworks). This means that we cannot put flat ini files in the project/instance root (as they won't be flat files but generated ones).
- If you'd like people that know things like paster to pick up our environment easily, I think, that: make it obvious where the deploy.ini & Co went and as they are generated files, leave a visible note at the beginning of the file, that states something like:
# NOTE: This file is generated by foo.recipe.paster, it will be # overwritten. Please apply changes to your buildout configuration # (located in XXX) instead of changing this file.
Yep. Thanks for all the input!
[snip: zopeproject and referencing 'instances']
If I try to conclude, then I see several decisions to make:
* "paster serve ..." vs. "commandname" approach.
It looks like that the "commandname" approach makes more sense with Zope as we certainly want to use the power of buildout and therefore have to use generated config files in 'exotic' (seen from the casual paster user's point of view) places.
* "commandname-dbg" vs. "commandname --debug"
I'll implement this concept in a next z3c.recipe.paster release. The idea behind the z3c.recipe.paster is to define several instances for what you need to have. This will also prevents to start a Zope instance in a debug mode if no debug option is defined. I'm not sure if this is the right ting for everyone, but I like to be restrictive on production systems.
This means, we have to decide, whether we want several commands performing special tasks (most probably: running everything in a special 'mode'), or one command that accepts options to indicate that special behaviour expected.
* "conffile.in" vs. buildout.cfg-embedded config files
I personally would really prefer the first approach for ease of use. If, however, this leads to problems with parsing the files, we are more or less bound to the latter. I'd like to investigate that further.
It looks like that the 'commandname-w/o-configfile-param' approach suits buildout-based environments best, which might be a good reason to be different in that respect from other paster-frameworks.
From my initial list of invokes (enriched by Hanno's Plone4 approach) then only some remain:
- "instance-fg <options>" and friends
- "mycommand [start|stop|status...] <options>"
The latter is partly already implemented by z3c.recipe.paster. Currently it does not support external config files and might do a bit more than needed (for example it calls ZConfig parser in schemaless mode, but there certainly is a good reason for it; in grokprojects we let zope.app.wsgi do this stuff), but it took me less than half an hour to create a flatter version that accepts both: internal and external config files (where external zope.confs are parsed with schemata) so that one can start a paster served grokproject with generated configuration files in parts/etc by doing::
$ bin/myapp
in foreground, where the script name is built from the buildout.cfg section name. It is actually a wrapper that runs::
paster serve buildout-set-ini-file [cmdline-options]
This means also, you have to do::
$ bin/myapp --daemon
to start the server in background (which is bad if you prefer SYSV compatible behaviour, i.e. ``bin/myapp start`` here), but you can do::
$ bin/myapp status
$ bin/myapp stop
I already have a zdaemon recipe which allows to use a zdaemon starting a paster setup. This looks like: [xpo] recipe = p01.recipe.zdaemon deployment = deploy program = ${buildout:directory}/bin/app zdaemon.conf = <runner> program ${buildout:directory}/bin/app daemon on socket-name /var/run/zdaemon/xpo.sock transcript /var/log/zdaemon/xpo.log # Enable this to run the child process as a different user user zope </runner> <eventlog> <logfile> path /var/log/zdaemon/xpo.log </logfile> </eventlog> This allows to point to a paster app which was setup with a z3c.recipe.paster. and is using a deploy buildout like: [deploy] recipe = zc.recipe.deployment name = zdaemon etc-directory = /etc/zdaemon log-directory = /var/log/zdaemon run-directory = /var/run/zdaemon logrotate-directory = /etc/logrotate.d crontab-directory = /etc/cron.d rc-directory = /etc/init.d user = zope Let me know if you are interested and like to move that to the zope repos. Regards Roger Ineichen
which does, what one expects.
One can also pass the usual paster options like ``--reload``. There is, however, no possibility to start in debug mode, i.e. with a debug.ini as argument, except you declare another command (at least I don't see any way to do it different).
It should also be possible to make use of z.r.paster in Plone4, as one can easily generate an ``instance-fg`` script this way. I wonder, however, whether this is still wanted with paster.
Hm, all this is confusing (and much too long). Maybe someone nevertheless has an opinion?
Best regards,
-- Uli
Hi there, Am Sonntag, den 19.04.2009, 09:33 +0200 schrieb Roger Ineichen:
Betreff: Re: [Zope-dev] Best way to invoke paster/WSGI apps on cmdline
Hi there,
Christian Theune wrote:
On Thu, 2009-04-16 at 01:01 +0200, Hanno Schlichting wrote:
Hi.
Sorry Hanno, your posting got lost on the way to my mailreader. I noticed you replied when reading Theuni's posting.
Uli Fouquet wrote:
As Grok is now approaching the 1.0 release we (i.e. some Grok developers and me) want to have a 'compliant' way to offer paster-based serving of Grok instances.
I'd love to have the same for Plone, but failed to find any canonical definition of "compliant" the same way you did.
Glad to hear that others have the same problems :)
Here's a short (probably incomplete) list of some 'full-stack' paster-capable 'frameworks' and how they invoke paster from the commandline (please correct/extend if I am wrong/incomplete here):
================ =================== ============================ Framework Ini-file location Paster call ================ =================== ============================ grokproject parts/etc/ bin/paster serve \ parts/etc/deploy.ini repoze.bfg <project root> paster serve MyProject.ini turbogears2 <project root> paster serve development.ini pylons <project root> paster serve development.ini zopeproject etc/ bin/paster serve \ etc/deploy.ini z3c.r.paster parts/<app>/ bin/app
I'll add the current Plone 4 way:
Plone parts/instance/etc bin/instance-fg
where instance-fg is a wrapper that calls:
bin/paster serve parts/instance/etc/paste.ini
We thought about this as well.
Some motivators for me, as I saw some of those variants around and some input with my experience from the buildout business:
- As a Unix-Lover, I do appreciate starting services having a SYSV style call, which means: (somecommand) start/stop/restart/...
Thus no calls that require me to remember a config file location and pass it on.
This also means that (somecommand) needs to be configurable as I might want to deploy multiple buildouts into the same machine having the deployment options create the service scripts in /etc/init.d and thus they can't all be called "instance". Having it named "myproject" seems better to me.
I see. I am not sure, everybody has this requirement, but we should keep it in mind. The question here is: how do you name your commands then? Do you opt for (somecommand) --debug or similar in that case?
- Also, I do like my autocomplete to leave me with full command name completions, thus: (somecommand)-ctl (somecommand)-debug doesn't work well for me: I always have to start typing again to complete the command *and* give a parameter. (Remember that I want parameters so I have SYSV compatible init commands right away)
- In a buildout environment, the use of .in files breaks the flexibility that the various buildout mechanism deliver to us: extends, variable inclusion, etc. have to be built again and again and again in the recipes. We started out with zope3instance recipes that did that but finally moved away from this.
Interesting. This, at least with Grok, can become a serious problem. We currently have five different configuration files generated by `grokproject`. Admitted, that one might skip one (`debug.ini`), we are left with four, which require three different configuration grammars:
zope.conf, zdaemon.conf: ZConf grammar deploy.ini: ConfigParser grammar site.zcml: ZCML
but this is a different topic.
Some of those files (namely the .ini files) meanwhile have grown large defining loggers and all that. Putting them into buildout.cfg makes it nearly unreadable. It becomes even difficult to check, which part of the configuration you're currently watching. People with more advanced setups might run into even greater trouble.
So, the pros and cons of external vs. buildout.cfg-embedded config files might read like this (for embedded configs):
* pros:
- supports all buildout mechanisms (namely: var interpolation and extend)
- all configs in one file (you know where to look for them)
* cons:
- harder to read
- confusing syntax (mix of at least three grammars)
- impossible to split permissions on parts of the config
With external configs it's the opposite picture. Can you tell more about the problems with zope3instance?
- Do not make me check in generated things.
Of course. As with buildout we _will_ have generated files (opposite to most other frameworks). This means that we cannot put flat ini files in the project/instance root (as they won't be flat files but generated ones).
- If you'd like people that know things like paster to pick up our environment easily, I think, that: make it obvious where the deploy.ini & Co went and as they are generated files, leave a visible note at the beginning of the file, that states something like:
# NOTE: This file is generated by foo.recipe.paster, it will be # overwritten. Please apply changes to your buildout configuration # (located in XXX) instead of changing this file.
Yep. Thanks for all the input!
[snip: zopeproject and referencing 'instances']
If I try to conclude, then I see several decisions to make:
* "paster serve ..." vs. "commandname" approach.
It looks like that the "commandname" approach makes more sense with Zope as we certainly want to use the power of buildout and therefore have to use generated config files in 'exotic' (seen from the casual paster user's point of view) places.
* "commandname-dbg" vs. "commandname --debug"
I'll implement this concept in a next z3c.recipe.paster release. The idea behind the z3c.recipe.paster is to define several instances for what you need to have.
This will also prevents to start a Zope instance in a debug mode if no debug option is defined. I'm not sure if this is the right ting for everyone, but I like to be restrictive on production systems.
This means, we have to decide, whether we want several commands performing special tasks (most probably: running everything in a special 'mode'), or one command that accepts options to indicate that special behaviour expected.
* "conffile.in" vs. buildout.cfg-embedded config files
I personally would really prefer the first approach for ease of use. If, however, this leads to problems with parsing the files, we are more or less bound to the latter. I'd like to investigate that further.
It looks like that the 'commandname-w/o-configfile-param' approach suits buildout-based environments best, which might be a good reason to be different in that respect from other paster-frameworks.
From my initial list of invokes (enriched by Hanno's Plone4 approach) then only some remain:
- "instance-fg <options>" and friends
- "mycommand [start|stop|status...] <options>"
The latter is partly already implemented by z3c.recipe.paster. Currently it does not support external config files and might do a bit more than needed (for example it calls ZConfig parser in schemaless mode, but there certainly is a good reason for it; in grokprojects we let zope.app.wsgi do this stuff), but it took me less than half an hour to create a flatter version that accepts both: internal and external config files (where external zope.confs are parsed with schemata) so that one can start a paster served grokproject with generated configuration files in parts/etc by doing::
$ bin/myapp
in foreground, where the script name is built from the buildout.cfg section name. It is actually a wrapper that runs::
paster serve buildout-set-ini-file [cmdline-options]
This means also, you have to do::
$ bin/myapp --daemon
to start the server in background (which is bad if you prefer SYSV compatible behaviour, i.e. ``bin/myapp start`` here), but you can do::
$ bin/myapp status
$ bin/myapp stop
I already have a zdaemon recipe which allows to use a zdaemon starting a paster setup. This looks like:
[xpo] recipe = p01.recipe.zdaemon deployment = deploy program = ${buildout:directory}/bin/app zdaemon.conf = <runner> program ${buildout:directory}/bin/app daemon on socket-name /var/run/zdaemon/xpo.sock transcript /var/log/zdaemon/xpo.log # Enable this to run the child process as a different user user zope </runner>
<eventlog> <logfile> path /var/log/zdaemon/xpo.log </logfile> </eventlog>
This allows to point to a paster app which was setup with a z3c.recipe.paster. and is using a deploy buildout like:
[deploy] recipe = zc.recipe.deployment name = zdaemon etc-directory = /etc/zdaemon log-directory = /var/log/zdaemon run-directory = /var/run/zdaemon logrotate-directory = /etc/logrotate.d crontab-directory = /etc/cron.d rc-directory = /etc/init.d user = zope
Let me know if you are interested and like to move that to the zope repos.
Very interesting. Thank you very much, Roger! Right now I have some problems with zdaemon as told in my reply to Christian. But if everybody likes it, also Grok/grokproject will go that way :) May I tell about my problems with z3c.recipe.paster as well? I know this sounds like complaining all the time, but I am also willing to help and contribute code if it helps. In general I am still attracted by z.r.paster :) For what I get from the discussion until now, I notice that everybody seems to agree, that Zope projects usually are different from other paster projects in that they use buildout and have some kind of meta-configuration. This means, that we have to generate for instance .ini files. Overall, this is not a problem but gives us the power of zc.buildout, which might also be attractive for other paster frameworks that still lack meta-configuration of that kind. If we don't want the 'genuine' paster approach ('paster serve ...') (because we want the power of buildout), then we might want to keep our way as open as possible for non-Zope frameworks, making buildout more attractive for others. What I mean by this is: if possible, one should avoid to require Zope-specific eggs (like ZConfig) in recipes installing paster wrappers. This blocks non-Zope pasters from using the recipe and everybody will start to do its own thing again. I think it is not necessary and further complicates setups (well, maybe it _is_ necessary, but I didn't get the concrete reason yet ;). It also means: appropriate recipes might focus on basic tasks and make no assumptions about the location or even the existance of 'zope.conf', 'site.zcml', etc. This should instead be set in the appropriate .ini file. The only thing I can think of, paster wrapper recipes should really be interested in, is an .ini file. So instead of extending z.r.paster to support more use-cases, I'd tend to flatten it and make it more simple, removing dependencies and the like. This is just a general line I'd like to follow and most probably I've overseen important pre-conditions or side-effects, so please correct me here. Best regards, -- Uli
Uli Fouquet wrote:
Very interesting. Thank you very much, Roger! Right now I have some problems with zdaemon as told in my reply to Christian. But if everybody likes it, also Grok/grokproject will go that way :)
Hhm, for Plone 4 I was so far considering to not use zdaemon at all anymore. Both mod_wsgi and supervisord should be able to handle the multiple process case, so we can get out of this business. Hanno
participants (4)
-
Christian Theune -
Hanno Schlichting -
Roger Ineichen -
Uli Fouquet