On Tue, 2002-11-26 at 01:29, Jamie Heilman wrote:

...

* where possible, the banishment of environment variables as configuration.

I have to say, I think this is a bad idea. What is it about environment variables do you find worthy of banishment? I can tell

"Banishment" was a bad word. It's not really so much that I think that environment variables are evil or anything, it's just that I think there should be one and only place to find and set most configuration data for Zope. Currently you configure Zope via a combination of z2.py switches and envvars. There are currently 40 Zope environment variables and 22 z2.py switches. There is no definitive reason for any one of these settings becoming a switch over an envvar or vice-vera (it was at the whim of the particular developer who decided to add the config requirement), so it appears (and is) rather haphazard. Some things like 'Z_DEBUG_MODE" and the -D switch actually even do the same thing, and it's hard to tell which you're supposed to use and which is supposed to "win" in these cases. Additionally, some things aren't really configurable at all (you're stuck with file or syslog logging, no way to send to an NT event service, etc). We'd like to fix this kind of thing and provide a more consistent front-end to configuring Zope settings. Additionally, since lots of Zope configuration data wants to be hierarchical (loggers, zodb databases, severs), and since environment variables and switches don't lend themselves particularly well to being hierarchical, a hierarchical config file format is nice. I hope this makes sense as a defense of using a config file. I don't particularly like config files either, but I think once a piece of software reaches a critical mass, they become a necessary evil. In my opinion, Zope reached that mass about three years ago. ;-)

a) Code spent garnering values is minimal, I don't have to worry about acres of code spent parsing yet another config file format or the inevitable quirks and bugs in that code. It doesn't get much simpler than os.environ.get()

Yup. We've got a seriously complicated piece of software to deal with config files now, and it's only getting more complicated. So I can sympathize with this entirely.

b) I have tools to examine runtime environments which are external to the zope application and any of its limitations. If I wanna know what a particular value is, I can just find out using ps. I don't have to dig around in the zmi, or rely on any custom clients to query that data from a running instance.

I feel that it's just as simple (or maybe simpler) to look in a config file. The Zope config file will be stored on the filesystem (not in the ZODB), and it's in a format much like an Apache config file, so no ZMI or custom client stuff will be necessary. There will be tools provided to parse the config file if necessary.

c) Environment variables lend themselves to nicely to modification by third party tools; I can set environment variables in shell scripts, python scripts, whatever--the paradigm is language agnostic. In particular I'm rather fond of using daemontools and its envdir program in conjunction with a custom z2.py replacement that lets me take advtanage of the excelent supervision and logging infrastucture provided by daemontools. Force me to use an external config file and its going to make this kind of thing more difficult to do.

That said I know they aren't an end-all solution for all configuration needs, but just because they don't fit every need I don't see why they should be completely discounted.

They shouldn't be. Lots of applications use environment variables as "superglobals" (e.g. $HOME, $PYTHONPATH, etc.), and this should be the kind of thing that Zope should use them for as well. I can see, for example, setting the SOFTWARE_HOME using an environment variable or throwing some debug switch using an envvar. But stuff like "ZSESSION_TIMEOUT_LIMIT" and "ZOPE_DTML_REQUEST_AUTOQUOTE" and the mishmash of logging parameters (STUPID_LOG_FILE, ZSYSLOG, ZSYSLOG_ACCESS, etc) are just begging to be put into some sort of configuration file. So far, we've done a bunch of work to come up with a config file format, some parsing software, and some configuration schema software. We're hoping that with a little work, we can make the configuration of Zope through a config file not suck *too* badly. If you have ideas about what should (and should not) be done wrt canonizing a config file for Zope, please detail specifics here as nothing is set in stone. Cheers, - C

On Tue, 2002-11-26 at 11:28, Jamie Heilman wrote:

Well you won't hear a peep out of me if z2.py gets the axe, as I mentioned I stopped using it long ago in favor of something that was only 39 lines and only did exactly what I needed and nothing more.

z2.py is a dead man walking. ;-)

I suppose, as movement in this direction is imminent, what I can probably get away with doing is just hooking all my custom environment variable stuff into the configuration machinery at startup. But thats assuming that once the configuration file is parsed the results are bundled into a single data structure that is then globally available to the rest of the zope machinery. As I haven't looked at the cvs branch in question perhaps you could tell me how valid this assumption is.

It currently works this way, but the location of the configuration(s) need to be canonized. - C

On Tue, 2002-11-26 at 05:22, Tino Wildenhain wrote:

Oh, while we are at it - how do I set an environment variable in a python script for use outside the script? E.g. like "export" does in shell scripts?

Not from a Zope Python Script (it's restricted by the security machinery) but from any other piece of Python: import os os.environ['NAME] = 'VALUE'

On Tue, 26 Nov 2002, Tino Wildenhain wrote:

Tried it:

...

...

...

os.environ['MYENVIRON']='FooBar' ^D

echo $MYENVIRON

^^^^ nothing.

What is wrong?

rdmurray@stage:~>cat temp.sh export TEST='abc' echo $TEST rdmurray@stage:~>sh temp.sh abc rdmurray@stage:~>echo $TEST ^^^^^ equally nothing. export (and the equivalent python above) sets the environment variable in the current process environment space, *not* the parent (that's a *much* trickier operation). The obverse (TEST='abc' somecommandhere) sets the environment variable only in the *sub*process environment space (the one being created) and does not affect the current process. The python equivalent of *that* would be to supply a modified *copy* of the ENVIRON dict to execve or similar. To do the python equivalent of what happens when you set a variable in a script *without* using export (the current process environment is changed but the change is not passed to subprocesses), you'd have to make a copy of the environ before modifying it, and pass *that* to execve or equivalent. --RDM

On Tue, 2002-11-26 at 02:47, Guido van Rossum wrote:

I'd like to compare notes on this. Where's the control script that you are using? I'm hoping that we can use (some future evolution of) zdaemon/{zdaemon,zdctl}.py in the Zope 2.7 trunk. This separates the concerns of

- the actual server process (Zope/ZEO/ZRS; for ZEO, this is runsvr.py)

- a daemon subprocess manager, which restarts the subprocess when it dies, etc. (zdaemon.py)

- a tool to control the subprocess manager, telling it to start/restart/stop the server subprocess, etc. (zdctl.py)

This sounds like a good arrangement. Most of the code for the "ctl" app that's checked in to the chrism-install-branch is in lib/python/Controller/ZctlLib.py. A "stub" which imports this code is written out to the "bin/zctl.py" file in an instance home (after you run configure; make; make instance, for example). The pedigree of this code is that it was originally coded up by Tres but it has been hacked by me since then to support Windows and to support the loading of configuration data from a config file with "matching" command-line parameters (mostly so you can inspect the config file based environment and pass in --switches that match the names of keys in the config file). It's not a particularly pretty piece of code and I wouldn't mind at all seeing it replaced.

...

* better support for Windows services

I'd like to hear more about this; my zdaemon.py currently doesn't do Windows at all. Where can I peek at your code?

A file named ntservice.py is written to the bin directory of an instance home after "make instance". To install an NT service based on an instance, you would call bin/ntservice.py install. There's not much more to control when it comes to NT services. They don't seem to require any of the coding that POSIX stuff does to switch effective users or run in the background, etc. I'm not sure that you can tell an NT service to restart a process that has died, but probably. This feature is not wired up to any installer or anything, but it should be.

...

* RPM-building support. * where possible, the banishment of environment variables as configuration.

All of these things save for the last are more or less complete on the "chrism-install-branch" of the Zope trunk. The major missing piece is the fleshing out of the interaction between the configuration system (ZConfig) and Zope itself. It works currently but it needs to be improved and canonized to everyone's liking.

Can you point to where Zope's use of ZConfig is?

chrism-install-branch in lib/python/zope.py and lib/python/Controller/Main.py. You can sorta see it in action if you create an instance home using the instructions that come with the branch and mess around with the config file in etc/zope.conf (put bad keys and sections in there, misspell things, etc, it should give you a pretty informative message about where in the config you screwed up). The sample config file is a little misleading (actually... parts of it are just plain wrong) as it details an older generation of config file ideas. I have not had time to regenerate the default config file. The best place to get an idea of what values are available is the schema for the Zope config in lib/python/Controller/directives.xml.

...

Since ZConfig will also used by other ZC software (ZEO, ZRS), there's a bit of back and forth that needs to be done to get everybody to agree on a way to do it. Once that is worked out, and the proposal is cleaned up, I think we can just merge.

Apologies in advance: my refrain seems to be "where's your code for X?" -- there's so much good stuff in your branch that I don't know where to begin, especially not with the Zope part (in ZODB, I know my way around better).

It's really no problem, I realize how difficult it is to reverse engineer somebody else's head. Hopefully I can improve the proposal soon and the vision might get clearer to other folks but me. I suppose I should add "better logging support" to this list of features as well, but since we've already got it going on the trunk and we're already figuring it out, I won't. - C

I gather from reading other posts that this is a different format than Zope 3's ZCML. Would it be worth it to at least make some sort of 'ZCML Lite' to do this sort of configuration? The main thing I'm concerned about is that there's a whole new Zope coming down the road that depends heavily on configuration files, and it might make sense to start weaning people onto its general syntax if Zope 2's finally going to have a config file.

There was a conscious decision to have two different "configuration" language: ZCML for *software* configuration, and ZConfig for *process* configuration. A sysop is not expected to have to edit ZCML, but she will definitely have to edit the ZConfig file. Since XML is not designed for routine editing/reading by humans (its "human-readability" is a debugging feature), ZConfig uses a more readable format. It's documented in the ZConfig CVS tree.

ZRS? This isn't the long-discussed replicated storage is it?

Yes, http://www.zope.com/Products/ZopeProducts/ZRS --Guido van Rossum (home page: http://www.python.org/~guido/)

...

...

...

...

"CM" == Chris McDonough <chrism@zope.com> writes:

>> * configure; make; make install installation >> I thought that there was an effort to use distutils, or to at >> least use a distutils-ish setup. What happened to that? CM> It does use distutils, owing to the work of Matt Behrens and CM> Shane on the setup.py file. The configure; make stuff just CM> wraps distutils to a large degree. And "configure; make; make install" is a worth goal. From a sysadmin's point of view, they're already conditioned to doing this to install open source software, so under the Pinball Machine Principle, when they see one in Zope, they'll run that first before reading (the whole) README file. Low barrier of entry is a good thing! -Barry

On Tue, 2002-11-26 at 13:56, Dieter Maurer wrote:

...

* where possible, the banishment of environment variables as configuration.

Please do not!

Do not what?