[Zope-dev] Thread safety in Zope 2 (was: [Zope-dev] zope 2 and non-multithread safe extensions) extensions)

[Jim Fulton]

Anthony Baxter wrote:
> 
> How should a Product that's not thread-safe indicate this to Zope 2?

It doesn't.  It must be thread safe.
 
> Say, for the sake of argument, that I had an (unreleased) SNMP product
> that wasn't thread-safe. How will this play in Zope2?

It will work incorrectly unless the number of application treads
is limited to one (e.g. 'python z2.py -t1').

This topic deserves some more discussion.  Which is the topic of this
note. :)

1. How can you tell if a product is thread safe?

   If a Zope product defines *persistent* objects and 
   those objects don't use any *mutable* gloal or shared variables
   (such as class attributes or default arguments), then the product
   is thread safe! *Pay close attention to this. It could let
   you off the hook!* :)

   A key feature of Zope 2 (ZODB 3) is that (a copy of) a 
   persistent object is never accessed by more than one thread
   (unless the programmer goes way out of thier way).  Each thread
   uses it's own database connection(s).  Each connection has it's
   own copies of persistent objects.  This feature provides a major 
   simplification for application developers.

   If a product *does* make use of mutable shared data, then the product
   must take steps to assure that they are used safely.  Some key examples
   of shared data include:

     - Variables defined in modules (globals),

     - Variables defined in classes,

     - Default arguments to Python functions.

   Note that a mutable global variable can be thread safe without
   special care in some cases.  For example, thanks to the methods
   append and pop, lists can be used as thread-safe stacks and 
   queues without any extra locks, because they are protected by the global
   interpreter lock.

2. What can I do if my product is not thread safe?

   I can think of two choices:

     a. Make it thread safe,

     b. Tell my "customers" that it can only be used in
        Zope applications with a single application thread.

3. How do I achiev thread safety for my objects?

   If they are persistent objects, you don't need to.
   If they are non-persistent objects that are *only*
   referenced by persistent objects, you also don't need to worry.

   This is a big topic, but in many cases, simple approaches
   can be used.

   First, you can assure that only one thread can call operations
   on your objects at one time and that data are accessed only through
   operations.  If your objects are of extension types, then the Python
   global interpreter lock protects operations on them (unless they release the
   lock).  If your objects are written in Python, you can protect them by
   mixing in the 'Synchronized' class from the 'Sync' module.

   The Synchronized class provides a per-instance lock that gets 
   automatically acquired and released on entry to and exit from 
   methods on your objects.

   Second, you should, if possible, provide a thread-safe API for your objects.
   In a thread-safe API, the operations are self-contained.  The meaning of 
   an operation depends on the history of operation calls.  An example 
   of a non-thread-safe API is the regex API.  Regex objects have a 
   'group' method whose results depend on results of previous 'search' and
   'match' calls.  

   If you can't change the API for an object to make it thread safe, then you
   can often create a wrapper for the object that is thread safe (see for example, 
   lib/python/ts_regex.py) or make sure that you only make thread safe calls.
   In many places in Zope, we make sure we only make thread safe calls on regex objects
   by storing references to the thread-safe methods rather than to compiled
   regular expressions themselves.

   If the approach sketched above doesn't work for you, then you need to
   put on your thinking cap and get up to speed on thread programming.

Jim
  
--
Jim Fulton           mailto:jim@digicool.com   Python Powered!        
Technical Director   (888) 344-4332            http://www.python.org  
Digital Creations    http://www.digicool.com   http://www.zope.org    

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.