[Zope-Checkins] CVS: Zope/doc - TODO.txt:1.1.2.1

Fred L. Drake, Jr. fred@zope.com
Tue, 25 Feb 2003 17:10:13 -0500


Update of /cvs-repository/Zope/doc
In directory cvs.zope.org:/tmp/cvs-serv16409

Added Files:
      Tag: new-install-branch
	TODO.txt 
Log Message:
Migrate the to-do list from the chrism-install-branch, with lots of
updates.  There's more to update, but we've actually come a long way.

This file should not be merged into the head.


=== Added File Zope/doc/TODO.txt ===
The following is a TODO list dealing with installation and startup
changes proposed for the Zope trunk.  It is organized into three
categories:

  "TODO BEFORE MERGE" (items to do before the chrism-install-branch,
  upon which most of these changes are now being made, can be merged
  into the Zope trunk)

  "BEFORE RELEASE OF ZOPE 2.7A" (items to do before the first alpha
  release of Zope 2.7)

  "MAYBE NEVER BUT NICE TO HAVE" (items that are not on any critical
  path)

-------------------------
    TODO FOR MERGE
-------------------------

Critical review of config file schema

  A config file is a contract with the user.  We want to get the Zope
  config file as right as possible the first time, as it will be very
  hard to change later.  One of the tricks to doing this is to make
  sure that as few keys and sections as possible in the config are
  "required", so that if we *do* need to change the schema for the
  config later it is likely not to impact a majority of users (as most
  of them will be sticking with the defaults).  We provide sane
  defaults for most things via default= in ZConfig schema and
  procedural code in ZConfig "handlers".  We would optimally like the
  default Zope config file to be able to be "empty" and still have
  Zope work.  This is currently not the case (software_home and
  instance_home are required keys).

Replace ZCtlLib/zopectl with zdctl variant

  Replace the current ZCtlLib/zopectl stuff with a variant of Guido's
  zdctl/runsrvr system, which we might call "zopectl".

  Issues:

    - I dont think we need a config file for zopectl.  We should give
      it sensible hardcoded defaults instead, and it should probably
      be capable of reading the Zope config file to find out the
      things it needs to know (logfile placement comes to mind).

    - zdctl/runsrvr doesn't do Windows.  It would be nice to have a
      zopectl on Windows but not strictly necessary.  If there is no
      zopectl on Windows, we will need to spell this out in the docs.
      We will also need the ntservice.py module to issue different
      commands to start and stop the service.

Deal with command-line-specified overrides to configuration values

  It is currently not possible to override config file-specifed values
  by using arguments to zctl or zope.py.  This will be required in
  reality.  There is a bunch of code that anticipates this possibility
  in zope.py and Zope.Startup.ZctlLib.py (anything that mentions
  "overrides" or "getSchemaKeys" is likely aiming to deal with this),
  but none of it actually works.

  XXX This was done somewhat via a cmdline.py module in Zope/Startup
  but needs to be replaced with zdoptions.

Write some unit tests

  Write unit tests for Zope.Startup packages.

Make sure ZConfig datatype conversion and handler errors are caught
the right way by runzope

  lib/python/zope.py is the responsible for causing the zconfig
  configuration file to be parsed, and needs to deal with errors in a
  sane way.  This is tricky because it now houses the top-level
  exception handler which needs to do sane things to distinguish
  between configuration errors, true system exit, and unknown errors.

Config file needs better inline docs

  The Zope ZConfig config file has some inline docs.  They need to be
  completed.  To centralize this data, the docs might be done in the
  schema itself.  A schema introspector which generates the default
  config file could be created.

Sanity check help and usage output of command-line programs

  Make sure runzope and zopectl have sane help output
  (make sure it matches the capabilities of the system).

Test 'effective user' support

  Great pains have been taken in configuration integration to defer
  database and logger construction until after Zope has had a chance
  to change effective users.  Make sure that no files are written to
  disk (maybe via unit tests) before Zope has had a chance to switch
  effective users.

Performing the merge

  All changed files are located on the new-install-branch; this
  includes additions and removals as well.  Each changed portion of
  the tree (which may be whole subtrees or individual files) needs to
  be merged into the trunk separately, and then testing and commit for
  the whole should be performed as a unit.

Review impact of missing Data.fs.in

  The branch install process doesn't copy the "Data.fs.in" file to an
  instance_home/var directory.  This file is the "initial" filestorage
  for Zope that gets copied to "Data.fs" during installation in every
  prior branch of Zope.  This file is evil and must die at some point,
  and most of the necessary code to set things up properly in the face
  of an empty Data.fs already exists in OFS.Application.initialize.
  Things seem to work fine without Data.fs.in but we should just give
  it a bit of review to see what gets left out, if anything.

  When we find that Data.fs.in isn't necessary, we need to remove it
  from the HEAD.

  XXX What is actually needed to verify that it isn't needed?

  AAA THe best way to do this is to write a unit test which verifies
      that the default objects installed into a "new" ZODB when Zope
      starts are present and working, and their content either matches
      the content of what would have come from Data.fs.in or is "close
      enough".  The code that installs these default objects exists in
      the initialize function within lib/python/OFS/Application.

Figure out why Zope complains about stale PythonScripts at startup

  After Zope is installed via the default installation routine, it
  complains about stale PythonScripts in some cases.  I haven't
  investigated this, but it needs to be fixed.

Notifying owners of automated testers

  Chris Withers and others who do automated tests of the HEAD will
  need to either discontinue the HEAD tests or they will need to make
  changes to their testers to build Zope properly using the new
  method.  This needs to be explained on zope-coders.

Review the Zope Book 2.6 Edition chapters and come up with revisions
or at least create a Zope Trunk Install HowTo

  The 2.6 edition Zope Book at
  http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition has
  three chapters which detail installation (installing and starting
  zope), maintenance (maintaining zope) and ZEO (scalability and ZEO).
  These chapters should be reviewed for inaccuracies with respect to
  the forthcoming trunk changes and changes should be made "offline"
  to allow a Zope Book 2.7 edition.

  At least create a HowTo which summarizes the differences between
  installing 2.6 and installing the trunk.

--------------------------------
  BEFORE RELEASE OF ZOPE 2.7A
--------------------------------

Default logging destinations

  If logging parameters are unspecified in the configuration, the
  default is to log zLOG messages to CLIENT_HOME/event.log and to log
  access messages to CLIENT_HOME/access.log.  This differs from older
  branches inasmuch as log messages in default mode (aside from
  startup messages) will no longer be written to stdout.
  STUPID_LOG_FILE and friends no longer have any effect.

  This is also why logtail is a desirable option (to give back this
  useful feature).

  This change needs to be communicated in docs and to developers in
  some way.


"configure" shell scripts need peer review

  I'd like someone to review the "configure" and "configure.bat"
  scripts in the top-level directory and suggest/make changes.  These
  scripts help find a suitable Python interpreter and create a
  Makefile on UNIX and Windows, respectively.

Update Zope distribution creation machinery

  The current distribution-creation machinery will almost certainly
  fail if it's not updated with knowledge of the new Zope installation
  regime.

Beautify output of "make install".

  The output of make install includes some spurious warnings from
  distutils.

Zope.Startup.nt.NTService.py and instance_home/bin/ntservice.py need
testing

  These modules allow you to associate an NT service with a Zope
  instance_home.  Success means being able to invoke the ntservice.py
  module to create an NT service which can successfully start and stop
  a Zope instance on a per-instance-home basis.

  Update: As of Jan. 13, this fails -- instance-home-based services
  report failure on start (even though they start) and cannot be
  stopped.

What to do about envvars?

  Envvars are still used "under the hood" in ZConfig handlers as the
  result of particular configuration declarations in order to make
  Zope do the right thing (e.g. INSTANCE_HOME, SOFTWARE_HOME,
  DTML_REQUEST_AUTOQUOTE, SESSION_TIMEOUT_MINUTES and other envvars
  are set in ZConfig handlers for their respective keys).  But envvars
  should not be used to try to configure Zope, as the handlers
  overwrite existing envvars with prejudice.  We need to come down on
  one side or the other about envvars.. either they should be
  respected at startup as they always have been or they should be
  explicitly not respected.  Currently they are not respected.

  We need to communicate this decision to developers and update
  doc/ENVIRONMENT.txt as necessary.

win32-eventlog needs testing

  The "win32-eventlog" log handler (which is creatable via the config
  file) needs to be tested.

RPM install should recompile .pycs

  The RPM spec file (inst/Zope.spec.in) should delete and recompile
  .pycs/.pyos during an RPM install to make sure tracebacks contain
  real Zope filenames instead of filenames relative to the build
  directory.  Alternately the RPM install should not contain .pycs or
  .pyos.

  XXX Not clear why this is actually needed; importing a module causes
  the __file__ to be updated with the right filename.  A nice touch,
  but not required.

  AAA Is this a new Python feature or has it always been this way?
  Tracebacks used to show the location of the .pyc where it had been
  compiled, not where it actually lived.

RPM install versioning scheme needs to be documented

  The RPM builder has a versioning scheme via the generated Makefile
  in order to allow for the install of multiple simultaneous versions
  of Zope on a single system.  This versioning scheme should be
  documented.

Deal with lack of PCGI

  In the old setup style, "w_pcgi.py" was invoked to install Zope with
  a PCGI ("persistent CGI") socket and files necessary for using Zope
  in conjunction with Apache/IIS.  There are no knobs for installation
  right now to do the same thing.  The 'pcgi' directory still exists
  in the Zope root dir and the 'make_resource.py' file in "inst" also
  is available, in case we decide to do something with them.

Server construction errors need to be better

  When a server is constructed, it attempts to bind to a port.  If the
  port cannot be bound, an error is raised.  The error currently doesn't
  include the port number, and should.


------------------------------
 MAYBE NEVER BUT NICE TO HAVE
------------------------------

ZConfig defaults

  We deferred several issues that we recognized as areas for
  improvement in ZConfig that might make it possible to avoid writing
  nasty procedural code for default handling unnecessary
  (e.g. Zope.Startup.handlers._RootHandler).  See
  http://my.zope.com/CPM/CPM/issues/4 and
  http://my.zope.com/CPM/CPM/issues/3.  Not necessary for merge, but
  useful to think about for future.