[Zope-dev] Zopeservice and sitecustomize

[Chris McDonough]

On Mon, 2004-03-08 at 20:54, Sake wrote:

> Everything works fine until I've learned that I can put 
> "sys.setdefaultencoding('cp874')" into sitecustomize.py

I assume you created a new sitecustomize.py to hold this in your
activestate Python's library directory, then?

>   I can start it manually from "runzope.bat", but it never 
> start through the windows system service. A full day investigation 
> reveal that the trouble cause by the missing of the 
> '<SOFTWARE_HOME>\lib\python' in the system environment's "PYTHONPATH".  
> The "runzope.bat" set that up before then execution of  
> "Zope.Startup.run.py", hence it run fine. But "zopeservice.py" rely on 
> the "<SOFTWARE_HOME>\bin\Lib\site-packages\sitecustomize.py" to set up 
> the correct "PYTHONPATH". Here is the code inside Zope's sitecustomize.py

This was put here because in my tests I could not make Python recognize
the "right" set of paths by setting an environment variable in the
zopeservice script.

As you probably noticed, Zope does not run in the same Python
interpreter instance as the one that zopeservice.py runs under. 
zopeservice.py instead turns around and invokes another Python
interpreter to actually do the running of Zope itself (that's what
"start_cmd" is set for).  Worse, the windows Service code actually
doesn't invoke zopeservice.py directly; it is run as a side effect of
the PythonService.exe executable being run.  Setting a PYTHONPATH via
os.environ in zopeservice was not doing the trick for me (at least on
Win2K); why was unclear, although I think it was because the environment
being modified by setting os.environ within zopeservice is not carried
over to child processes.  A batch file couldn't be used as the target
for the service because the Windows shell doesn't keep track of child
processes in the same way UNIX does: child processes aren't killed when
their parents are and there is no way to 'exec' another command from a
Windows batch file.

> Unluckily, this sitecustomize.py is now masked with my sitecustomize.py 
> inside Activestate's site-package directory, which actually get loaded 
> by Zope via the Python registry load path 
> (HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.3\PythonPath) instead 
> of the expected one.

I don't think I understand why this happens.  That's not surprising. 
There is some complicated DWIM that Windows Python does when it starts
up to try to guess what should go on sys.path and in what order.  Tim
Peters dug up a comment in Python's code and sent it to me a bit ago
about why this happens.  It can be found in Python's PC/getpathp.c file:

   PATH RULES FOR WINDOWS:
   This describes how sys.path is formed on Windows.  It describes the
   functionality, not the implementation (ie, the order in which these
   are actually fetched is different)

   * Python always adds an empty entry at the start, which corresponds
     to the current directory.

   * If the PYTHONPATH env. var. exists, it's entries are added next.

   * We look in the registry for "application paths" - that is, sub-keys
     under the main PythonPath registry key.  These are added next (the
     order of sub-key processing is undefined).
     HKEY_CURRENT_USER is searched and added first.
     HKEY_LOCAL_MACHINE is searched and added next.
     (Note that all known installers only use HKLM, so HKCU is typically
     empty)

   * We attempt to locate the "Python Home" - if the PYTHONHOME env var
     is set, we believe it.  Otherwise, we use the path of our host
.EXE's
     to try and locate our "landmark" (lib\\os.py) and deduce our home.
     - If we DO have a Python Home: The relevant sub-directories (Lib,
       plat-win, lib-tk, etc) are based on the Python Home
     - If we DO NOT have a Python Home, the core Python Path is
       loaded from the registry.  This is the main PythonPath key,
       and both HKLM and HKCU are combined to form the path)

   * Iff - we can not locate the Python Home, have not had a PYTHONPATH
     specified, and can't locate any Registry entries (ie, we have
_nothing_
     we can assume is a good path), a default path with relative entries
is
     used (eg. .\Lib;.\plat-win, etc)

  The end result of all this is:
  * When running python.exe, or any other .exe in the main Python
directory
    (either an installed version, or directly from the PCbuild
directory),
    the core path is deduced, and the core paths in the registry are
    ignored.  Other "application paths" in the registry are always read.

  * When Python is hosted in another exe (different directory, embedded
via
    COM, etc), the Python Home will not be deduced, so the core path
from
    the registry is used.  Other "application paths" in the registry are
    always read.

  * If Python can't find its home and there is no registry (eg, frozen
    exe, some very strange installation setup) you get a path with
    some default, but relative, paths.

>From this (and without a Windows machine in front of me), I can't really
make any sense out of why your Activestate Python's sitecustomize.py is
being found instead of Zope's Python sitecustomize.py if you're running
Zope using the Zope Python install.  I suspect it may be because of
placement of your sitecustomize.py file and the rule named "We look in
the registry for "application paths"", but that's a guess.

> I don't think setting up PYTHONPATH inside sitecustomize.py is a good 
> idea. Better keep this mechanism for site specific problems.  I'd rather 
> insert a line into zopeservice.py like this.
> 
> import os.path
> from os.path import dirname as dn
> import sys
> 
> # these are replacements from mkzopeinstance
> PYTHONW = r'C:\Zope-2.7.0\bin\pythonw.exe'
> SOFTWARE_HOME=r'C:\Zope-2.7.0\lib\python'
> INSTANCE_HOME = r'C:\Zope-MIB'
> ZOPE_HOME = r'C:\Zope-2.7.0'
> 
> ZOPE_RUN = r'%s\Zope\Startup\run.py' % SOFTWARE_HOME
> CONFIG_FILE= os.path.join(INSTANCE_HOME, 'etc', 'zope.conf')
> PYTHONSERVICE_EXE=r'%s\bin\PythonService.exe' % ZOPE_HOME
> 
> sys.path.insert(0, SOFTWARE_HOME)
> os.environ['PYTHONPATH'] = SOFTWARE_HOME  <---------------- inserted line
> 
> from nt_svcutils.service import Service
> 
> servicename = 'Zope_%s' % str(hash(INSTANCE_HOME))
> 
> class InstanceService(Service):
>     start_cmd = '"%s" "%s" -C "%s"' % (PYTHONW, ZOPE_RUN, CONFIG_FILE)
>     _svc_name_ = servicename
>     _svc_display_name_ = 'Zope instance at %s' % INSTANCE_HOME
>     _exe_name_ = PYTHONSERVICE_EXE
> 
> if __name__ == '__main__':
>     import win32serviceutil
>     win32serviceutil.HandleCommandLine(InstanceService)
> 
> 
> 
> This is much more palatable in my opinion.

Does this work?  On all of W2K, Win98, and WinXP?  If so, I'd be glad to
ditch the current munging of sitecustomize.py.  As I said, it wasn't
working for me, at least on Win2K.

- C