[Zope] Re: Global variables in ExternalMethod modules

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ben Last (Zope) wrote:

| I'm trying to work out what the rules are for global (that is,
| module-level) variables in a module that contains external methods.
|
| The scenario: we have a single module (.py file) in the Extensions
| subdirectory of the instance directory, which contains all our external
| methods plus many helper methods.  Amongst other things, this module
| contains a lot of methods that perform database queries to an associated
| MySQL database.  The same module is also used by non-Zope Python tasks,
| so the queries are all direct, not via Zope SQL products.
|
| As part of the ongoing optimization of the code, I've been looking at
| having a single MySQLdb connection per Zope thread, and re-using this
| for all SQL queries.  My understanding was that if I declare a
| connection variable at module level, like so:
|
| databaseConnection = None

If this *were* a true module (see below), you would not be getting
per-thread semantics from it;  you would need to keep a mapping keyed by
thread ID to get such semantics.

| I could then initialize it in a method that's called at the start of
| handling every request, but re-use the value between requests.  However,
| it *appears* that the value of the databaseConnection variable is
| (nearly) always None when any new request starts, as though the module
| were being reloaded afresh for every one.  Thus there seems to be no way
| to keep per-thread module-level variables.
|
| This is on our test platform, where the Zope instance is running in
| debug-mode.
|
| We've also noticed an odd connection leak on our production server,
| where Zope is not running in debug-mode.  It *appears* as though the
| __del__ methods of MySQLdb.Connection objects are not being called (and
| thus the connections are remaining open indefinitely).  We've recently
| started wrapping Connection objects in a class of our own, and tracing
| __init__ and __del__ calls for that also appears to bear out that theory.
|
| All this on Zope 2.7.3, Python 2.3.4, Linux.
|
| So; is there a quick-n-easy summary of how module-level variables behave
| in ExtensionMethod modules?  Is there any reason that, in a non-debug
| environment, the __del__ methods of simple Python objects may not be
| called?
|
| Regards to all

The files containing ExternalMethod implementations are not "modules",
really, and so your expectations about module-level globals are not
being met.  Instead, the EM machinery "execs" the text of the EM file in
a custom namespacee, and then extracts the code object from it.

I think you are headed into "build a product" territory here, unless you
want your EM to monkey-patch its cache into an existing module somewhere.

Tres.
- --
===============================================================
Tres Seaver                                tseaver at zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB/7yiGqWXf00rNCgRAmA/AJwMeG9ESBmk0xls8FLeYXWR4GRBOQCdGJUN
123OT26+4dvVAguLuJ5k8so=
=cMTR
-----END PGP SIGNATURE-----