Proposed installation changes for review
Hi folks, An effort to improve Zope 2 installation, startup, and configuration for Zope 2.7 has been ongoing for some time now (see http://dev.zope.org/Wikis/DevSite/Proposals/InstallationAndConfiguration). The effort has reached a point where we believe it is reasonable to merge its fruits to the Zope trunk. Before doing so, however, we want to gather input from people who care deeply about Zope configuration and installation; particularly those who have significant investments in the existing behaviors of Zope 2 configuration and installation. A bunch of stuff is changed (most notably Zope no longer uses environment variables as configuration, and instead defers to command-line arguments and config file values). What follows is a summary of changes, a walk-through of a Zope 2 installation under this regime, and a list of the configuration-file changeable parameters exposed by the current Zope configuration schema. Please provide your feedback before this Friday, March 14. If possible, check out and try the new software. If there are showstopper issues for a lot of people, we will consider not performing the merge directly after that date. Otherwise, the merge will be performed at any time after the 14th. Thanks! - C ----- Major changes in Zope 2.7 ========================= Zope 2.7 will contain a number of changes related to the build process and configuration. The scripts used to start Zope will also be substantially changed. The implementation of these changes is not yet complete, but a substantial portion has been done and is expected to be merged into the Zope 2 trunk in the near future. The exact schedule for merging these changes has not been determined; additional notice will be provided. These changes are substantially based on efforts by Chris McDonough on the "chrism-install-branch". Major changes ------------- The following are the most important and user-visible changes compared to Zope 2.6 and the current CVS HEAD: - Configuration is normally done by editing a config file instead of passing command line options. The configuration file is handled by the ZConfig package. - Environment variables are no longer used for configuration. - Building and installing the software have become more clearly distinct; the installation can be separate from the build. - Zope 2.7 uses the "instance home" model in the normal case: a script in the installed software is used to create new "instances" of Zope. Each instance is defined by a directory tree containing configuration, scripts, and data. (For convenience, the instance and the installation can be located in the same directory, and the build area may be used in lieu of an installation if desired.) - The z2.py, start, and stop scripts have been replaced by the runzope and zopectl scripts. - The runzope/zopectl scripts do not accept the same command-line arguments as z2.py used to accept. - The wo_pcgi.py script used to build Zope without PCGI support has been replaced by the common Unix build sequence of configure, make. - The pcgi/ directory and the w_pcgi.py script have been removed from the top of the source tree. The pcgi/ directory may be made available as a separate distribution. (The application server still has support to allow connections from a PCGI-capable Web server.) - The ZServer package has been moved from the top of the source tree into the lib/python/ directory. Getting the code today ---------------------- The implementation lives in the "new-install-branch", which is a "partial" branch rather than a complete branch: only the files and directories which need to be changed have been placed on the branch; all other files are used from the CVS HEAD. The current state of this code can be checked out from CVS using the command: cvs -d :pserver:anonymous@cvs.zope.org:/cvs-repository co \ -r new-install-branch -f Zope The installation instructions in doc/INSTALL.txt has been updated on the branch; that file provides detailed information on creating an installation and instance. (A copy has been attached for convenience.) A sample installation/startup runthrough --------------------------------------- The below is the output from a sample install session. During this session, we install the Zope software to /tmp/zopehome and install an instance home to /tmp/insthome. [chrism@james software]$ cd NewInstallBranch/ [chrism@james NewInstallBranch]$ ./configure --prefix=/tmp/zopehome Configuring Zope installation Testing for an acceptable Python interpreter... Python version 2.1.3 found at /home/chrism/bin/python Python version 2.1.3 found at /home/chrism/bin/python2.1 Python version 2.2.2 found at /home/chrism/bin/python2.2 The optimimum Python version (2.2.2) was found at /home/chrism/bin/python2.2. - Zope top-level binary directory will be /tmp/zopehome. - Makefile written. Next, run make. [chrism@james NewInstallBranch]$ make "/home/chrism/bin/python2.2" inst/setup.py -q build_ext -i <... compilation output elided ...> Zope built. Next, do 'make install' (or 'make instance' to run a Zope instance directly from the build directory). [chrism@james NewInstallBranch]$ make install <... installation output elided ...> Zope binaries installed successfully. Now run '/tmp/zopehome/bin/mkzopeinstance' [chrism@james NewInstallBranch]$ /tmp/zopehome/bin/mkzopeinstance mkzopeinstance requires exactly one argument mkzopeinstance: Create a Zope instance home. usage: mkzopeinstance [options] directory Options: -h/--help -- print this help text -u/--user NAME:PASSWORD -- set the user name and password of the initial user [chrism@james NewInstallBranch]$ /tmp/zopehome/bin/mkzopeinstance \ --user=admin:123 /tmp/inst [chrism@james NewInstallBranch]$ cd /tmp/inst/ [chrism@james inst]$ ls bin etc Extensions import inituser log Products README.txt var [chrism@james inst]$ bin/zopectl start . daemon process started, pid=2947 [chrism@james inst]$ bin/zopectl stop . daemon process stopped Changing the configuration via the config file ---------------------------------------------- The Zope configuration file uses the ZConfig API and file format. See the ZConfig documentation PDF at http://cvs.zope.org/Packages/ZConfig/doc/zconfig.pdf for more information. The default Zope configuration file will live in the instance_home's /etc directory and will be named 'zope.conf'. The Zope configuration file allows you to change the following configuration variables: instance home client home debug mode effective user production installation (on/off) locale number of zserver threads the python "check interval" (thread switch timing) zserver read only mode pid file name lock file name structured text header level publisher profile file (debugging) cgi environment dns server (hostname resolving in logfiles) ip address (for zserver bindings) http realm automatic quotation of dtml request data security manager stack size security policy implementation (C, Python) skip authentication checking skip ownership checking max number of session objects session "add notify" script path session "delete notify" script path session timeout minutes access rule suppress site root suppression database quota size read only database zeo client name logging (access, event, debug, filenames, to syslog, etc). network servers (http, dav, ftp, monitor, etc) databases (storages, database classes, etc.)
A few questions / concerns listed below, otherwise it looks fine to me... On Mon, Mar 10, 2003 at 04:41:48PM -0500, Chris McDonough wrote:
debug mode
does this still toggle a whole bunch of things?
production installation (on/off)
what's this mean?
network servers (http, dav, ftp, monitor, etc)
is this where you set the ports? One thing I didn't see in your list is a way to extend the products path (currently I do that with $PRODUCTS_PATH). We need that. One more question: Does "zopectl.py" always detach from the terminal (booo) or not (yay) ? Or does it behave like z2.py and this depends on "debug mode" (booo) ? -- Paul Winkler http://www.slinkp.com
On Mon, 2003-03-10 at 18:07, Paul Winkler wrote:
A few questions / concerns listed below, otherwise it looks fine to me...
On Mon, Mar 10, 2003 at 04:41:48PM -0500, Chris McDonough wrote:
debug mode
does this still toggle a whole bunch of things?
Yes. It's still the case that nobody really knows what debug mode means. ;-) We just switched from only being able to specify the unknown via a command-line switch to being able to switch it from a config file and the command line.
production installation (on/off)
what's this mean?
Sorry, that should have been "product installation". This is the same as what the FORCE_PRODUCT_LOAD envvar does now.
network servers (http, dav, ftp, monitor, etc)
is this where you set the ports?
Yes.
One thing I didn't see in your list is a way to extend the products path (currently I do that with $PRODUCTS_PATH). We need that.
This wasn't documented in ENVIRONMENT.txt so it never made it in. Thanks for catching it.
One more question: Does "zopectl.py" always detach from the terminal (booo) or not (yay) ?
No. It can detach from the terminal, but it needn't. It is actually a mini shell. [chrism@james inst]$ bin/zopectl program: /tmp/inst/bin/runzope daemon manager not running zdctl> ? Documented commands (type help <topic>): ======================================== fg foreground help kill logreopen logtail quit reload restart shell show start status stop wait Undocumented commands: ====================== EOF zdctl> fg export EVENT_LOG_FILE EVENT_LOG_FILE= /tmp/inst/bin/runzope HTH, - C
On Mon, 2003-03-10 at 17:07, Paul Winkler wrote:
A few questions / concerns listed below, otherwise it looks fine to me...
On Mon, Mar 10, 2003 at 04:41:48PM -0500, Chris McDonough wrote:
debug mode
does this still toggle a whole bunch of things?
production installation (on/off)
what's this mean?
network servers (http, dav, ftp, monitor, etc)
is this where you set the ports?
One thing I didn't see in your list is a way to extend the products path (currently I do that with $PRODUCTS_PATH). We need that.
One more question: Does "zopectl.py" always detach from the terminal (booo) or not (yay) ? Or does it behave like z2.py and this depends on "debug mode" (booo) ?
Actually I like the way z2.py detaches or doesn't detach. Perhaps a separate config option would be good to control this. -- Edward Muller Interlix - President Web Hosting - PC Service & Support Custom Programming - Network Service & Support Phone: 417-862-0573 Cell: 417-844-2435 Fax: 417-862-0572 http://www.interlix.com
Edward Muller writes:
Actually I like the way z2.py detaches or doesn't detach. Perhaps a separate config option would be good to control this.
Detaching, or "daemonizing", will be a separate configuration parameter from everything else. The basic mechanism will be that provided by the zdctl.py/zdrun.py scripts in the zdaemon package. -Fred -- Fred L. Drake, Jr. <fred at zope.com> PythonLabs at Zope Corporation
On Mon, Mar 10, 2003 at 10:31:13PM -0500, Fred L. Drake, Jr. wrote:
Detaching, or "daemonizing", will be a separate configuration parameter from everything else.
great, that is exactly what i really want. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's GARGANTUAN SKULL OF THE REVOLUTION! (random hero from isometric.spaceninja.com)
- Environment variables are no longer used for configuration.
I'll say it one more time. The roadmap[1] states under the "Simplifying the Zope experience" section: * simple tasks should be simple! Now, code required to extract a value from the environment: import os try: value = sanitize(os.environ.get("SOMETHING", default)) except Unsanitary: ...cope... # where 'sanitize' is in reference to any conversion procedures required # prior to the use of 'value' by program code Pretty simple. Enter ZConfig: $ du -sk ZConfig 374 ZConfig 374k of work devoted to replacing os.environ.get and its sanitizing code, and the result is a XML config file. I'm not saying this all for naught, but really, it should give you pause. Just how much have you really simplified configuration by doing this? Personally I think the problem of Zope's configuration hassles has been mis-identified. ZConfig cleans up and centralizes a maze of badly strewn out configuration code. It will extend the lifetime of the �ber-server concept ZServer employs. It does nothing to prevent the same mess from happening again, down the road. I believe a more lasting approach is to seperate the servers into small individual programs which only do 1 thing, and do it well. Then you combine those servers under a unified mangement framework, connect them together by using common communication idioms, the idea being--and follow me here, to use small programs combined together to provide a larger service. Sound familiar? -- Jamie Heilman http://audible.transient.net/~jamie/ "...thats the metaphorical equivalent of flopping your wedding tackle into a lion's mouth and flicking his lovespuds with a wet towel, pure insanity..." -Rimmer [1] http://dev.zope.org/Resources/ZopeRoadmap.html
On Mon, 2003-03-10 at 19:51, Jamie Heilman wrote:
- Environment variables are no longer used for configuration.
I'll say it one more time.
The roadmap[1] states under the "Simplifying the Zope experience" section:
* simple tasks should be simple!
Now, code required to extract a value from the environment:
import os try: value = sanitize(os.environ.get("SOMETHING", default)) except Unsanitary: ...cope...
# where 'sanitize' is in reference to any conversion procedures required # prior to the use of 'value' by program code
I'm sure you know this, but you are oversimplifying the situation. There are 41 (known) individual environment variables that control Zope's runtime configuration. Each use of an environment variable has its own parsing code, its own error-handling code and the actual code that does the work (ex: the session envvar parsing stuff in OFS/Application.py). If we take your example and make the error handling code "real" and the work code "real" we can estimate that it will consume about 1k. Multiply the number of bytes it contains with 41 and we can approximate about 41k of ad-hoc envvar handling code in Zope now. That's a boatload of largely untested and decentralized code, all of which is doing configuration. There is nothing simple about it. ;-)
Pretty simple. Enter ZConfig: $ du -sk ZConfig 374 ZConfig
374k of work devoted to replacing os.environ.get and its sanitizing code, and the result is a XML config file. I'm not saying this all for naught, but really, it should give you pause. Just how much have you really simplified configuration by doing this?
I couldn't really guess how much code in Zope is devoted to runtime configuration right now because it's spread over the entire codebase. I suspect you're right that it's not 374k. But for ZConfig only 116k is code, the rest is docs and tests: there are neither (to speak of, discounting the laughable ENVIRONMENT.txt) in the current 2.6/trunk codebase for configuration.
Personally I think the problem of Zope's configuration hassles has been mis-identified. ZConfig cleans up and centralizes a maze of badly strewn out configuration code. It will extend the lifetime of the Über-server concept ZServer employs. It does nothing to prevent the same mess from happening again, down the road.
Actually, it does. Packages may declare their own config schema type definitions and they may be included in the context of a larger configuration schema. This is demonstrated in the Zope schema in the new-install-branch in lib/python/Zope/Startup/zopeschema.xml: <schema prefix="Zope.Startup.datatypes" datatype=".root_config" handler="root_handler"> <!-- type definitions --> <import package="zLOG"/> <import package="ZODB"/> <import package="ZServer"/> .... We import the schema type definitions from the zLOG, ZODB, and ZServer packages here (these are named component.xml in each of these packages). The zope schema file uses these definitions to compose its own (type-checked) schema for a config file, and they can be (and will be) reused for ZEO and ZC's ZRS (Zope Replication Server). For Zope, ZServer is just another package that happens to define the schema type definitions for network servers.
I believe a more lasting approach is to seperate the servers into small individual programs which only do 1 thing, and do it well. Then you combine those servers under a unified mangement framework, connect them together by using common communication idioms, the idea being--and follow me here, to use small programs combined together to provide a larger service. Sound familiar?
I think this is done already. ZConfig is very general and very generic; you can configure just about anything with it and it has no Zope dependencies whatsoever. Fred Drake wrote it so it's pretty solid too. Before dismissing it out of hand, I'd encourage you to try it out. - C
Chris McDonough wrote:
Before dismissing it out of hand, I'd encourage you to try it out.
I'm not dismissing it, and I think you need to go back and read what I wrote again very very carefully without reading anything into it. I'm not trying to imply that using environment variables to configure the current codebase will reduce the code footprint. Even if it did, because of the distributed nature of the technique, its damnedly hard to maintain in a project as large as Zope. Also, I didn't say ZConfig was 374k of code, I said it was 374k of *work*. I chose that word very carefully, and obviously thats going to err on the side of conservatism as certainly the work was not isolated to that single directory tree. The point I'm trying to make is that Zope has learned nothing from the UNIX philosophy. Yes, you can extend the config schema. You can grow new, better config files, of extraordinary magnitude. The all-powerful server will grow from being all-powerful to being all-powerful + n. It will be able to read mail. Its heralds shall sit upon mountain high throwns hewn of the finest O'Reilly and New Riders scripture. But lo, still you won't be able to do something as mundane as limit the memory the FTP server is able to consume without affecting the HTTP server. Fracture the server infrastructure into small, seperate processes. The configuration of the individual pieces becomes trivial. The understanding of the overall data flow improves. When there's nothing left to remove from code, you've won. Some of the breaks have already been made, like the separation of the storage from its front-end. Thats good, we need more action along those lines. -- Jamie Heilman http://audible.transient.net/~jamie/ "Paranoia is a disease unto itself, and may I add, the person standing next to you may not be who they appear to be, so take precaution." -Sathington Willoughby
On Mon, 2003-03-10 at 21:42, Jamie Heilman wrote:
Chris McDonough wrote: The point I'm trying to make is that Zope has learned nothing from the UNIX philosophy. Yes, you can extend the config schema. You can grow new, better config files, of extraordinary magnitude. The all-powerful server will grow from being all-powerful to being all-powerful + n. It will be able to read mail. Its heralds shall sit upon mountain high throwns hewn of the finest O'Reilly and New Riders scripture. But lo, still you won't be able to do something as mundane as limit the memory the FTP server is able to consume without affecting the HTTP server.
Point taken.
Fracture the server infrastructure into small, seperate processes. The configuration of the individual pieces becomes trivial. The understanding of the overall data flow improves. When there's nothing left to remove from code, you've won. Some of the breaks have already been made, like the separation of the storage from its front-end. Thats good, we need more action along those lines.
I think this is a noble goal, but far beyond the scope of the current project, which is stated on the project page as "Make it easier for sysadmins and new Zope developers to install and configure Zope". I believe that ZConfig will help us towards this goal. It should also not hinder us from reaching the goal that you've defined above. Thanks, - C
The point I'm trying to make is that Zope has learned nothing from the UNIX philosophy. Yes, you can extend the config schema. You can grow new, better config files, of extraordinary magnitude. The all-powerful server will grow from being all-powerful to being all-powerful + n. It will be able to read mail. Its heralds shall sit upon mountain high throwns hewn of the finest O'Reilly and New Riders scripture. But lo, still you won't be able to do something as mundane as limit the memory the FTP server is able to consume without affecting the HTTP server.
Fracture the server infrastructure into small, seperate processes. The configuration of the individual pieces becomes trivial. The understanding of the overall data flow improves. When there's nothing left to remove from code, you've won. Some of the breaks have already been made, like the separation of the storage from its front-end. Thats good, we need more action along those lines.
You're barking up the wrong tree. Zope 2 won't change. Zope 3 is still in a state of flux, and that's where you should aim your speech. --Guido van Rossum (home page: http://www.python.org/~guido/)
But lo, still you won't be able to do something as mundane as limit the memory the FTP server is able to consume without affecting the HTTP server.
You can do this with Zope. Just use ZEO and run one ZEO front-end for HTTP and one for FTP. -- Steve Alexander
Steve Alexander wrote:
But lo, still you won't be able to do something as mundane as limit the memory the FTP server is able to consume without affecting the HTTP server.
You can do this with Zope. Just use ZEO and run one ZEO front-end for HTTP and one for FTP.
Sure, but then you carry along all the baggage of 2 zserver instances. Its a start, but there's still a ways to go. -- Jamie Heilman http://audible.transient.net/~jamie/ "I was in love once -- a Sinclair ZX-81. People said, "No, Holly, she's not for you." She was cheap, she was stupid and she wouldn't load -- well, not for me, anyway." -Holly
Hopefully I'll get to play with this later on in the week.. But here are the things I know I need as part of my business: -The ability to specify which python to use to build, install and run zope with. Our production machines have various version 1.5.2, 2.1.X, 2.2.X, etc installed. I must be able to specify the location. -The ability for each instance to have separate Product directories. Some users want plone, some don't. Some develop their own software. -The ability to start/stop zope without access to /proc (or the equivalent on other unices) on Linux. I don't know if the changes made affect any of the above ... but it would suck major #$W%% if it did. Oh, and part of my business if hosting zope ... if you are wondering where the above (off the top of my head list) comes from. Other than that it looks like the changes will be good for me. Oh and some way to have multiple zope versions installed at the same time would be great as well. :-) On Mon, 2003-03-10 at 15:41, Chris McDonough wrote:
Hi folks,
An effort to improve Zope 2 installation, startup, and configuration for Zope 2.7 has been ongoing for some time now (see http://dev.zope.org/Wikis/DevSite/Proposals/InstallationAndConfiguration). The effort has reached a point where we believe it is reasonable to merge its fruits to the Zope trunk. Before doing so, however, we want to gather input from people who care deeply about Zope configuration and installation; particularly those who have significant investments in the existing behaviors of Zope 2 configuration and installation. A bunch of stuff is changed (most notably Zope no longer uses environment variables as configuration, and instead defers to command-line arguments and config file values).
What follows is a summary of changes, a walk-through of a Zope 2 installation under this regime, and a list of the configuration-file changeable parameters exposed by the current Zope configuration schema.
Please provide your feedback before this Friday, March 14. If possible, check out and try the new software. If there are showstopper issues for a lot of people, we will consider not performing the merge directly after that date. Otherwise, the merge will be performed at any time after the 14th.
Thanks!
- C
----- Major changes in Zope 2.7 =========================
Zope 2.7 will contain a number of changes related to the build process and configuration. The scripts used to start Zope will also be substantially changed. The implementation of these changes is not yet complete, but a substantial portion has been done and is expected to be merged into the Zope 2 trunk in the near future.
The exact schedule for merging these changes has not been determined; additional notice will be provided. These changes are substantially based on efforts by Chris McDonough on the "chrism-install-branch".
Major changes -------------
The following are the most important and user-visible changes compared to Zope 2.6 and the current CVS HEAD:
- Configuration is normally done by editing a config file instead of passing command line options. The configuration file is handled by the ZConfig package.
- Environment variables are no longer used for configuration.
- Building and installing the software have become more clearly distinct; the installation can be separate from the build.
- Zope 2.7 uses the "instance home" model in the normal case: a script in the installed software is used to create new "instances" of Zope. Each instance is defined by a directory tree containing configuration, scripts, and data. (For convenience, the instance and the installation can be located in the same directory, and the build area may be used in lieu of an installation if desired.)
- The z2.py, start, and stop scripts have been replaced by the runzope and zopectl scripts.
- The runzope/zopectl scripts do not accept the same command-line arguments as z2.py used to accept.
- The wo_pcgi.py script used to build Zope without PCGI support has been replaced by the common Unix build sequence of configure, make.
- The pcgi/ directory and the w_pcgi.py script have been removed from the top of the source tree. The pcgi/ directory may be made available as a separate distribution. (The application server still has support to allow connections from a PCGI-capable Web server.)
- The ZServer package has been moved from the top of the source tree into the lib/python/ directory.
Getting the code today ----------------------
The implementation lives in the "new-install-branch", which is a "partial" branch rather than a complete branch: only the files and directories which need to be changed have been placed on the branch; all other files are used from the CVS HEAD. The current state of this code can be checked out from CVS using the command:
cvs -d :pserver:anonymous@cvs.zope.org:/cvs-repository co \ -r new-install-branch -f Zope
The installation instructions in doc/INSTALL.txt has been updated on the branch; that file provides detailed information on creating an installation and instance. (A copy has been attached for convenience.)
A sample installation/startup runthrough ---------------------------------------
The below is the output from a sample install session. During this session, we install the Zope software to /tmp/zopehome and install an instance home to /tmp/insthome.
[chrism@james software]$ cd NewInstallBranch/ [chrism@james NewInstallBranch]$ ./configure --prefix=/tmp/zopehome
Configuring Zope installation
Testing for an acceptable Python interpreter...
Python version 2.1.3 found at /home/chrism/bin/python Python version 2.1.3 found at /home/chrism/bin/python2.1 Python version 2.2.2 found at /home/chrism/bin/python2.2
The optimimum Python version (2.2.2) was found at /home/chrism/bin/python2.2.
- Zope top-level binary directory will be /tmp/zopehome. - Makefile written.
Next, run make.
[chrism@james NewInstallBranch]$ make "/home/chrism/bin/python2.2" inst/setup.py -q build_ext -i <... compilation output elided ...>
Zope built. Next, do 'make install' (or 'make instance' to run a Zope instance directly from the build directory).
[chrism@james NewInstallBranch]$ make install <... installation output elided ...>
Zope binaries installed successfully. Now run '/tmp/zopehome/bin/mkzopeinstance'
[chrism@james NewInstallBranch]$ /tmp/zopehome/bin/mkzopeinstance mkzopeinstance requires exactly one argument
mkzopeinstance: Create a Zope instance home.
usage: mkzopeinstance [options] directory
Options:
-h/--help -- print this help text -u/--user NAME:PASSWORD -- set the user name and password of the initial user
[chrism@james NewInstallBranch]$ /tmp/zopehome/bin/mkzopeinstance \ --user=admin:123 /tmp/inst [chrism@james NewInstallBranch]$ cd /tmp/inst/ [chrism@james inst]$ ls bin etc Extensions import inituser log Products README.txt var [chrism@james inst]$ bin/zopectl start . daemon process started, pid=2947 [chrism@james inst]$ bin/zopectl stop . daemon process stopped
Changing the configuration via the config file ----------------------------------------------
The Zope configuration file uses the ZConfig API and file format. See the ZConfig documentation PDF at http://cvs.zope.org/Packages/ZConfig/doc/zconfig.pdf for more information.
The default Zope configuration file will live in the instance_home's /etc directory and will be named 'zope.conf'. The Zope configuration file allows you to change the following configuration variables:
instance home client home debug mode effective user production installation (on/off) locale number of zserver threads the python "check interval" (thread switch timing) zserver read only mode pid file name lock file name structured text header level publisher profile file (debugging) cgi environment dns server (hostname resolving in logfiles) ip address (for zserver bindings) http realm automatic quotation of dtml request data security manager stack size security policy implementation (C, Python) skip authentication checking skip ownership checking max number of session objects session "add notify" script path session "delete notify" script path session timeout minutes access rule suppress site root suppression database quota size read only database zeo client name logging (access, event, debug, filenames, to syslog, etc). network servers (http, dav, ftp, monitor, etc) databases (storages, database classes, etc.)
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Edward Muller
Interlix - President Web Hosting - PC Service & Support Custom Programming - Network Service & Support Phone: 417-862-0573 Cell: 417-844-2435 Fax: 417-862-0572 http://www.interlix.com
On Mon, 2003-03-10 at 22:13, Edward Muller wrote:
Hopefully I'll get to play with this later on in the week..
But here are the things I know I need as part of my business:
-The ability to specify which python to use to build, install and run zope with. Our production machines have various version 1.5.2, 2.1.X, 2.2.X, etc installed. I must be able to specify the location.
Yup. ./configure --with-python=/path/to/your/python
-The ability for each instance to have separate Product directories. Some users want plone, some don't. Some develop their own software.
This is inherent in INSTANCE_HOME behavior. The result of "mkzopeinstance" will contain a Products directory which may contain Zope software. Many instances can be configured to run from a single software home (a software home is the result of "make install")
-The ability to start/stop zope without access to /proc (or the equivalent on other unices) on Linux.
Yes.
I don't know if the changes made affect any of the above ... but it would suck major #$W%% if it did. Oh, and part of my business if hosting zope ... if you are wondering where the above (off the top of my head list) comes from. Other than that it looks like the changes will be good for me.
Oh and some way to have multiple zope versions installed at the same time would be great as well. :-)
Sure. In a Zope 2.7.0 source directory: ./configure --prefix=/opt/zope-2.7.0 make make install In a Zope 2.7.1 source directory: ./configure --prefix=/opt/zope-2.7.1 make make install ... then 'mkzopeinstance' from either. - C
Okay here is another one :-) given... ./configure --prefix=/opt/zope-2.7.0 make make install Once zope is installed in /opt/zope-2.7.0 can it be moved without damaging the install .... say to /home/virtual/some.host.name/opt/zope-2.7.0 ? In our hosting setup some things get run in a chroot, some things can't... Currently zope get's installed in a chroot environment for anyone who wants a zope install. It must be a complete install since when the user restarts it he will be in his chroot environment. So I'd ideally like to install zope in a way where all of the core of zope is in one place ... say ... /opt/zope/<version #> (/opt/zope/2.7.0, /opt/zope/2.7.1, etc...) This I can hardlink/symlink into each chroot and make permissions 755 root/root.
From there I would like to be able to install an 'instance', which is ... in my case meaning the data.fs, /Products directory, log files, etc, etc. The stuff that make this users instance theirs. When the install is happening, the script executing it would most likely be outside of the chroot ... but I guess it could be configured to chroot as well..
I already have start/stop scripts to go through the users that have a zope install and chroot into that users 'host' and then start zope as that 'hosts' administrative user. On Mon, 2003-03-10 at 21:47, Chris McDonough wrote:
On Mon, 2003-03-10 at 22:13, Edward Muller wrote:
Hopefully I'll get to play with this later on in the week..
But here are the things I know I need as part of my business:
-The ability to specify which python to use to build, install and run zope with. Our production machines have various version 1.5.2, 2.1.X, 2.2.X, etc installed. I must be able to specify the location.
Yup.
./configure --with-python=/path/to/your/python
-The ability for each instance to have separate Product directories. Some users want plone, some don't. Some develop their own software.
This is inherent in INSTANCE_HOME behavior. The result of "mkzopeinstance" will contain a Products directory which may contain Zope software. Many instances can be configured to run from a single software home (a software home is the result of "make install")
-The ability to start/stop zope without access to /proc (or the equivalent on other unices) on Linux.
Yes.
I don't know if the changes made affect any of the above ... but it would suck major #$W%% if it did. Oh, and part of my business if hosting zope ... if you are wondering where the above (off the top of my head list) comes from. Other than that it looks like the changes will be good for me.
Oh and some way to have multiple zope versions installed at the same time would be great as well. :-)
Sure.
In a Zope 2.7.0 source directory:
./configure --prefix=/opt/zope-2.7.0 make make install
In a Zope 2.7.1 source directory:
./configure --prefix=/opt/zope-2.7.1 make make install
... then 'mkzopeinstance' from either.
- C
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Edward Muller
Interlix - President Web Hosting - PC Service & Support Custom Programming - Network Service & Support Phone: 417-862-0573 Cell: 417-844-2435 Fax: 417-862-0572 http://www.interlix.com
On Tue, 2003-03-11 at 00:24, Edward Muller wrote:
Once zope is installed in /opt/zope-2.7.0 can it be moved without damaging the install .... say to /home/virtual/some.host.name/opt/zope-2.7.0 ?
Yes. Its location is only meaningful to the instance files that need to find it.
In our hosting setup some things get run in a chroot, some things can't...
Currently zope get's installed in a chroot environment for anyone who wants a zope install. It must be a complete install since when the user restarts it he will be in his chroot environment.
So I'd ideally like to install zope in a way where all of the core of zope is in one place ... say ... /opt/zope/<version #> (/opt/zope/2.7.0, /opt/zope/2.7.1, etc...)
This I can hardlink/symlink into each chroot and make permissions 755 root/root.
I think this will work. The only thing that might be a little weird is tracebacks generated by pyc files, as they may report the filenames of the Python modules where they were originally installed, instead of where they live now. There is some contention about whether this happens under Python 2.2, but I know it's true for Python 2.1 and prior.
From there I would like to be able to install an 'instance', which is ... in my case meaning the data.fs, /Products directory, log files, etc, etc. The stuff that make this users instance theirs. When the install is happening, the script executing it would most likely be outside of the chroot ... but I guess it could be configured to chroot as well..
You would need to chroot the run of makeinstance currently as it encodes paths to software within the instance files that start Zope. So if you ran it outside the chroot it would work, but when the user logged in to the chroot, the paths to the software would be wrong. I think this might be made configurable with a switch to mkzopeinstance (--sw_location=/some/path), though. I will add this to the tentative TODO, thanks.
I already have start/stop scripts to go through the users that have a zope install and chroot into that users 'host' and then start zope as that 'hosts' administrative user.
These scripts will unfortunately need to change for Zope 2.7 unless we create some sort of backwards compatibilty layer for startup. - C
On Tue, 2003-03-11 at 08:48, Chris McDonough wrote:
On Tue, 2003-03-11 at 00:24, Edward Muller wrote:
Once zope is installed in /opt/zope-2.7.0 can it be moved without damaging the install .... say to /home/virtual/some.host.name/opt/zope-2.7.0 ?
Yes. Its location is only meaningful to the instance files that need to find it.
In our hosting setup some things get run in a chroot, some things can't...
Currently zope get's installed in a chroot environment for anyone who wants a zope install. It must be a complete install since when the user restarts it he will be in his chroot environment.
So I'd ideally like to install zope in a way where all of the core of zope is in one place ... say ... /opt/zope/<version #> (/opt/zope/2.7.0, /opt/zope/2.7.1, etc...)
This I can hardlink/symlink into each chroot and make permissions 755 root/root.
I think this will work. The only thing that might be a little weird is tracebacks generated by pyc files, as they may report the filenames of the Python modules where they were originally installed, instead of where they live now. There is some contention about whether this happens under Python 2.2, but I know it's true for Python 2.1 and prior.
Well I can install zope in /opt/zope/2.7.1 (in the real root) and then when I symlink/hardlink it into a virtual host I can link it into that hosts /opt/zope/2.7.1 ... So that's not a biggie....
From there I would like to be able to install an 'instance', which is ... in my case meaning the data.fs, /Products directory, log files, etc, etc. The stuff that make this users instance theirs. When the install is happening, the script executing it would most likely be outside of the chroot ... but I guess it could be configured to chroot as well..
You would need to chroot the run of makeinstance currently as it encodes paths to software within the instance files that start Zope. So if you ran it outside the chroot it would work, but when the user logged in to the chroot, the paths to the software would be wrong.
That's not a problem ... at least IIRC. I can chroot when creating the account in a shell script and execute custom setup scripts.
I think this might be made configurable with a switch to mkzopeinstance (--sw_location=/some/path), though. I will add this to the tentative TODO, thanks.
all thought that would be nice.
I already have start/stop scripts to go through the users that have a zope install and chroot into that users 'host' and then start zope as that 'hosts' administrative user.
These scripts will unfortunately need to change for Zope 2.7 unless we create some sort of backwards compatibilty layer for startup.
Yeah. Oh well. They aren't that complex. :-) I wouldn't worry about the backward compatibility layer myself. I don't know if there is a great value add to it, aside from keeping users from going 'WTF happened?' :-) -- Edward Muller Interlix - President Web Hosting - PC Service & Support Custom Programming - Network Service & Support Phone: 417-862-0573 Cell: 417-844-2435 Fax: 417-862-0572 http://www.interlix.com
We currently rely on two scripts for running our Zopes: 1. A modified version of zctl.py. I originally got it from a moribund wiki on the Zope site. The main changes we've made have been to better separate parameters for Zope clients from parameters for the ZEO server and to run an additional server of our own along side the ZEO server (a simple distributed RAM Cache invalidation server). 2. A very simple sysv-init script that implements everything by calling the correct zctl.py. It looks like the new install and startup world will be a huge improvement over the current setup on the whole. I like moving the log files to their own directory. The things that seemed to be missing from your writeup were: Almost no mention of ZEO (only one mention of a "zeo client name" parameter). How does ZEO fit into this? I don't see an equivalent to "./zctl.py debug" anywhere. This starts up an interactive Python as a ZEO client with ZServer and Zope imported and app = Zope.app(). I use it constantly. Please?
The things that seemed to be missing from your writeup were:
Almost no mention of ZEO (only one mention of a "zeo client name" parameter). How does ZEO fit into this?
ZEO client configuration is included in the new configuration schema. ZEO server configuration has its own schema and tools, also based on ZConfig and the new zdaemon; you can check these out in the ZODB 3.2 alpha release. --Guido van Rossum (home page: http://www.python.org/~guido/)
--On Tuesday, March 11, 2003 10:17:08 AM -0500 Guido van Rossum <guido@python.org> wrote:
ZEO client configuration is included in the new configuration schema.
ZEO server configuration has its own schema and tools, also based on ZConfig and the new zdaemon; you can check these out in the ZODB 3.2 alpha release.
Does this mean that Zope 2.7 will require ZODB 3.2 for ZEO users? As an aside, am I the only one who's confused by this new bundling of ZEO as part of the stand alone ZODB product? Dan Pierson
ZEO client configuration is included in the new configuration schema.
ZEO server configuration has its own schema and tools, also based on ZConfig and the new zdaemon; you can check these out in the ZODB 3.2 alpha release.
Does this mean that Zope 2.7 will require ZODB 3.2 for ZEO users?
Zope 2.7 will *come with* ZODB 3.2 (or later, depending on the timing of the Zope 2.7 release), so I suppose so.
As an aside, am I the only one who's confused by this new bundling of ZEO as part of the stand alone ZODB product?
I don't know. What's confusing for you? The new approach is that: Zope includes ZODB includes ZODB What's confusing about that? --Guido van Rossum (home page: http://www.python.org/~guido/)
--On Tuesday, March 11, 2003 02:39:55 PM -0500 Guido van Rossum <guido@python.org> wrote:
Does this mean that Zope 2.7 will require ZODB 3.2 for ZEO users?
Zope 2.7 will *come with* ZODB 3.2 (or later, depending on the timing of the Zope 2.7 release), so I suppose so. ...
As an aside, am I the only one who's confused by this new bundling of ZEO as part of the stand alone ZODB product?
I don't know. What's confusing for you? The new approach is that:
Zope includes ZODB includes ZODB
What's confusing about that?
Zope doesn't (currently) include ZEO so to get an up to date ZEO we now have to obtain the correct version of the stand-alone ZODB and extract the contained ZEO. The version numbers of the ZEO and stand-alone ZODB are unrelated so figuring out which ZODB we need to get ZEO is a bit of a pain. Also, ZEO is packaged differently in the stand alone ZODB than it used to be in on it's own. Dan Pierson
Zope doesn't (currently) include ZEO so to get an up to date ZEO we now have to obtain the correct version of the stand-alone ZODB and extract the contained ZEO. The version numbers of the ZEO and stand-alone ZODB are unrelated so figuring out which ZODB we need to get ZEO is a bit of a pain. Also, ZEO is packaged differently in the stand alone ZODB than it used to be in on it's own.
That's why we're including the correct versions of ZODB and ZEO in Zope itself. That's already the case in Zope 2.6. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, 2003-03-11 at 10:01, Dan L. Pierson wrote:
The things that seemed to be missing from your writeup were:
Almost no mention of ZEO (only one mention of a "zeo client name" parameter). How does ZEO fit into this?
Configuration parameters are available for the ZEO ClientStorage storage within the config file.
I don't see an equivalent to "./zctl.py debug" anywhere. This starts up an interactive Python as a ZEO client with ZServer and Zope imported and app = Zope.app(). I use it constantly. Please?
Yes, I do too! ;-) Good catch. I've added this to the TODO... Thanks, - C
I seem to have a lot of problems with the init.d/zope script, since it keeps hanging when I do /etc/init.d/zope restart or during boot, which is pretty painful. In the debugging I see : 2003-03-11T17:48:47 PROBLEM(100) Hotfixes Not applying Hotfix_2002-06-14. The f ix has already been merged into Zope. Please uninstall the hotfix product. ------ 2003-03-11T17:48:47 PROBLEM(100) Init Ambiguous name for method of Products.Plug Ins.PlugIns.PlugInGroup: "manage_workspace" != "manage_main" ------ 2003-03-11T17:48:47 PROBLEM(100) Init Ambiguous name for method of Products.Logi nManager.LoginMethods.BasicCookieLogin: "manage_propertiesForm" != "manage_main" ------ 2003-03-11T17:48:54 INFO(0) ZODB Opening database for mounting: '144274512_10460 91853.725337' ------ 2003-03-11T17:48:54 INFO(0) ZODB Mounted database '144274512_1046091853.725337' at /temp_folder ------ 2003-03-11T17:48:54 INFO(0) Zope Default Object Creation /usr/lib/zope/import/Ex amples.zexp examples import file could not be found. ------ 2003-03-11T17:48:54 INFO(0) Plone: Active Pak 0.1 and HTMLTALParser patch 1.4 have NOT been applied (unneeded for Z ope 2.6+) ------ 2003-03-11T17:48:54 PROBLEM(100) Zope No such file or directory: /usr/lib/zope/l ib/python/Products/CMFCalendar/help ------ 2003-03-11T17:48:56 PROBLEM(100) Zope No such file or directory: /usr/lib/zope/l ib/python/Products/LoginManager/help ------ 2003-03-11T17:48:57 PROBLEM(100) Zope No such file or directory: /usr/lib/zope/l ib/python/Products/PlugIns/help ------ 2003-03-11T17:48:57 PROBLEM(100) Zope No such file or directory: /usr/lib/zope/lib/python/Products/ZPatterns/help ------ 2003-03-11T17:48:58 INFO(0) ZServer FastCGI Server (V1.0) started at Tue Mar 11 17:48:58 2003 IP : Port : 9673 Socket path : None Greetz, Johan
apologies for this cross-post: in case anyone's confused and/or intrigued, this discussion leaked into zope@zope.org from a discussion on zope-dev. i've set reply-to back to zope-dev. On Tue, Mar 11, 2003 at 11:29:56AM -0500, Chris McDonough wrote:
On Tue, 2003-03-11 at 10:01, Dan L. Pierson wrote:
The things that seemed to be missing from your writeup were:
Almost no mention of ZEO (only one mention of a "zeo client name" parameter). How does ZEO fit into this?
Configuration parameters are available for the ZEO ClientStorage storage within the config file.
I don't see an equivalent to "./zctl.py debug" anywhere. This starts up an interactive Python as a ZEO client with ZServer and Zope imported and app = Zope.app(). I use it constantly. Please?
Yes, I do too! ;-) Good catch. I've added this to the TODO...
Thanks,
- C
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
-- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's HEMI COBRA OF DOOM! (random hero from isometric.spaceninja.com)
On Tue, Mar 11, 2003 at 10:01:53AM -0500, Dan L. Pierson wrote:
I don't see an equivalent to "./zctl.py debug" anywhere. This starts up an interactive Python as a ZEO client with ZServer and Zope imported and app = Zope.app(). I use it constantly. Please?
+1. I also use zctl.py debug nearly every day. of course it's just a convenience, but it's an important convenience because nearly every document i can find on debugging Zope says roughly "...and of course you can use ZEO, but that's beyond the scope of this article..." -- Paul Winkler http://www.slinkp.com
participants (9)
-
Chris McDonough -
Dan L. Pierson -
Edward Muller -
Fred L. Drake, Jr. -
Guido van Rossum -
Jamie Heilman -
johan -
Paul Winkler -
Steve Alexander