[Zope-dev] Problem with zope.sendmail in Python >= 2.5.1

[Gary Poster]

Hi all.  One of the contributors to the Launchpad project has  
identified a problem with zope.sendmail and thread changes in Python  
 >= 2.5.1.  Here is his description:

"""
Description of the situation:

Prior to Python 2.5.1, the atexit handlers were executed when the  
*main* thread exits. However,http://bugs.python.org/issue1566280 was  
then filed, the essence of which is that the atexit handlers could  
tear down cross-thread resources that were still in use by other  
threads.

For this reason, the shutdown procedure was modified in Python 2.5.1 -  
now, when the main thread exits, it first makes a private call to  
threading._shutdown(), which waits until all non-daemon threads have  
exited, and *only then* proceeds to run the atexit handlers.

Herein lies the problem. zope.sendmail.delivery.QueueProcessorThread  
attempts to use atexit to notify itself when it should shut down.  
However, the Python runtime >= 2.5.1 will wait for the  
QueueProcessorThread to exit before it calls the atexit handlers!

Potential solutions:

(1) To gain behaviour most similar to earlier Python versions, an evil  
monkeypatch into Python internals like this: http://bazaar.launchpad.net/~maxb/zope3/launchpad-3.4-py2.5/revision/9works 
.

(2) Make the QueueProcessorThread a daemon thread. It will be  
terminated without notification when the interpreter exits, but the  
current QueueProcessorThread does not attempt to ensure its queue is  
completely flushed before shutdown anyway, so that should not matter.

I will attach a small Python demo program useful for illustrating and  
exploring the issue.
"""

You can see the bug report, and his demo program, here:

https://bugs.edge.launchpad.net/zope3/+bug/413335

I wasn't going to send this until I had investigated this myself, but  
I have been taking too long and thought I'd at least send this out to  
see if anyone has any thoughts.  Otherwise, I'll look into it ASAP and  
report back.

Thanks

Gary