[Zope] RE: Zope and mod_python

4 Feb 2005

      Thanks for the quick reply.

A first very simple ApacheBench looked promising : 
+ Apache2/mod_python with default number of Apache threads, etc
vs
+ Zope instances with default number of threads
shows that Apache2/mod_python is about 2 times faster.
But this does not prove lots yet, as default number of Apache threads is
high.

I will try to optimize my zhandler code, specially the ZPublisher response
(string containing the whole HTTP response) to mod_python response (I have
to read HTTP headers and HTTP body to correctly set the reponse). I will
probably just extend mod_python response object to be Zope Publisher
compliant.

I got the logging working (I mean the event log, cause of course Z2.log is
useless as I don't run any ZServer).
Well, each mod_python interpreter running one Zope handler print to the same
log file, but at least it works.

About the clean shutdown : as I don't run any ZServer, maybe there is
nothing particular to do ?
I have been looking at LifeTime.py code and it looks like shutdown code is
only for ZServers.

New version of __init__.py :
__all__ = ['zhandler']

zope_pythonpath='/export/home/pperegri/20050203/opt/Zope-2.7.4-0/lib/python'
zope_conf='/export/home/pperegri/test/conf/zhandler.conf'

import sys
if not zope_pythonpath in sys.path:
    sys.path.append(zope_pythonpath)

import Zope
if not Zope._began_startup:
    import os
    Zope.configure('/export/home/pperegri/test/conf/zhandler.conf')
    import App.config
    cfg = App.config.getConfiguration()
    if cfg.eventlog is not None:
        cfg.eventlog()
    import zLOG
    zLOG.LOG('zHandler', zLOG.INFO,'Init : starting Zope')
    Zope.startup()

Pascal

-----Message d'origine-----
De : Tres Seaver [mailto:tseaver@zope.com]
Envoyé : vendredi 4 février 2005 17:40
À : Pascal Peregrina
Objet : Re: Zope and mod_python

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Pascal Peregrina wrote:
| Hi,
|
| I have successfully (I think) integrated Zope with Apache/mod_python, just
| to make some tests.

Wow!  That is awesome.

| I would like to know if you think that this can be interesting in terms of
| performance (specially number of simultaneous requests being handled,
| compared to one "real" Zope instance, even with number of threads
| increased).

Good questions:  we would need to do a fair amount of difficult testing
to know for sure.  Certainly it is a big win for shops which are already
comfortable with managing large-scale dynamic Apache installations.

| Sorry in advance if I missed something important and if this is not
| interesting at all :)

Not a all.

| By now I have not been able to get the logging working,

Zope's logging relies on having a single "gateway" in the appserver
process which is used by the worker threads.  I think your mod_python
child processes are like Zope worker threads without the "parent" (which
has the gateway).  I wonder if configuring it to use syslog would Just
Work (TM)?

| and I am not
| completely sure about clean shutdown when Apache is stopped, but I will
| continue investigating...

Doesn't Apache / mod_python allow you to register a "graceful shutdown"
callback?

| Software :
| + httpd-2.0.52
| + mod_python-3.1.3
| + Python-2.3.4
| + psyco-1.4 (only to use Zope Speedpack product)
| + Zope 2.7.4
| + Several Zope products (Localizer, CMF 1.5.0, ...)
|
| Architecture :
| + One Zeo instance
| + Apache / mod_python
| and that's all :)
|
| I created a zhandler package (very first draft) containing :
| + __init__.py : Zope init
| + zhandler.py : the handler itself (it uses ZPublisher)
|
| Then, from standard build/install/configuration of all the software above,
| this is the configuration changes I did :
| + Created a Zeo instance
| + httpd.conf (global settings) :
| SetHandler mod_python
| PythonInterpreter zinterpreter
| PythonHandler zhandler.zhandler
| PythonDebug On
| + and this is the zope.conf file (I called it zhandler.conf) I used
| (basically it only consisted in removing all <server> occurences) :
|
############################################################################
| ###
| # Welcome to Zope 2.
|
############################################################################
| ###
| #
| # This is the Zope configuration file.  The Zope configuration file
| # shows what the default configuration directives are, and show
| # examples for each directive.  To declare a directive, make sure that
| # you add it to a line that does not begin with '#'.  Note that comments
| # are only allowed at the beginning of a line: you may not add comments
| # after directive text on the same line.
|
| # ZConfig "defines" used for later textual substitution
|
| %define INSTANCE /export/home/pperegri/20050203/instances/zope_rw
| %define ZOPE /export/home/pperegri/20050203/opt/Zope-2.7.4-0
|
| instancehome $INSTANCE
| products /export/home/pperegri/20050203/opt/Products
| debug-mode off
| zeo-client-name zcmd
|
| <eventlog>
|   level all
|   <logfile>
|     path $INSTANCE/log/cmdline_event.log
|     level info
|   </logfile>
| </eventlog>
|
| <logger access>
|   level WARN
|   <logfile>
|     path $INSTANCE/log/cmdline_Z2.log
|     format %(message)s
|   </logfile>
| </logger>
|
| <zodb_db temporary>
|     mount-point /temp_folder
|     <temporarystorage>
|       name sessions
|     </temporarystorage>
|     container-class Products.TemporaryFolder.TemporaryContainer
| </zodb_db>
|
| <zodb_db remote>
|     mount-point /
|     cache-size 40000
|    <zeoclient>
|      cache-size 256MB
|      server 192.168.11.163:9001
|      read-only off
|      storage 1
|      name zeostorage
|      var $INSTANCE/var
|    </zeoclient>
| </zodb_db>
|
|
| + __init.py__
| __all__ = ['zhandler']
|
|
zope_pythonpath='/export/home/pperegri/20050203/opt/Zope-2.7.4-0/lib/python'
| zope_conf='/export/home/pperegri/test/conf/zhandler.conf'
|
| import sys
| if not zope_pythonpath in sys.path:
|     sys.path.append(zope_pythonpath)
|
| import Zope
| if not Zope._began_startup:
|     Zope.configure('/export/home/pperegri/test/conf/zhandler.conf')
|     Zope.app()
|
|
| + zhandler.py
| import mod_python.apache
| import os
| import sys
|
| # this will save memory
| #os.environ = {}
|
| import ZPublisher
|
| class ZopeStdout(mod_python.apache.NullIO):
|
|     """
|     Class that allows writing to the socket directly for Zope.
|     """
|
|     def __init__(self, req):
|         self.req = req
|         self.pos = 0
|
|     def write(self, result):
|         if not result or self.pos: return
|         pos1=result.find('\r\n\r\n')
|         pos2=result.find('\n\n')
|         if pos1!=-1 and pos2!=-1 and pos1>pos2:
|             ss = result.split('\n\n', 1)
|         else:
|             ss = result.split('\r\n\r\n', 1)
|             if len(ss) < 2:
|                 ss = result.split('\n\n', 1)
|         ss[0]=ss[0].replace('\r\n', '\n')
|         for line in ss[0].split('\n'):
|             if line.find(':')!=-1:
|                 h,v=line.split(':',1)
|                 if h.lower() == "status":
|                     status = int(v.split()[0])
|                     self.req.status = status
|                 elif h.lower() == "content-type":
|                     self.req.content_type = v
|                     self.req.headers_out[h] = v
|                 else:
|                     self.req.headers_out.add(h, v)
|         if len(ss)==2:
|             self.req.write(ss[1])
|             self.pos+=len(ss[1])
|
|     def tell(self): return self.pos
|
| def handler(req):
|     save_env, si, so = mod_python.apache.setup_cgi(req)
|     sys.stdout = ZopeStdout(req)
|     if 'REQUEST_URI' in os.environ:
|         os.environ['PATH_INFO']=os.environ['REQUEST_URI'].split('?')[0]
|         os.environ['PATH_TRANSLATED']=os.environ['PATH_INFO']
|     if 'HTTP_CONNECTION' in os.environ:
|         os.environ['CONNECTION_TYPE']=os.environ['HTTP_CONNECTION']
|     os.environ['SCRIPT_NAME']=''
|     names=os.environ.keys()
|     for name in names:
|         if not name.startswith('HTTP_') and not
name.startswith('PATH_') and
| not name.startswith('SERVER_') and not name in
|
['CONNECTION_TYPE','GATEWAY_INTERFACE','REQUEST_METHOD','REMOTE_ADDR','SCRIP
| T_NAME','QUERY_STRING']:
|             del os.environ[name]
|     try:
|         ZPublisher.publish_module('Main', stdin=sys.stdin,
|                                   stdout=sys.stdout, stderr=sys.stderr,
|                                   environ=os.environ)
|     finally:
|         mod_python.apache.restore_nocgi(save_env, si, so)
|     return mod_python.apache.OK

Very impressive work.

Tres.
- --
===============================================================
Tres Seaver                                tseaver@zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCA6VZGqWXf00rNCgRAkhcAJ4m6pDA5q6CkKeYyg2IkXUNLgBc0wCfWWy0
npGh1X7k3Q8kqLmJLfq8q6w=
=zdNn
-----END PGP SIGNATURE-----