oddity in MailHost queue processor thread
Hi all, I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary. It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once. Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails. regards, # un monkey patch du mailhost de zope2.11, ce dernier utilise une thread # par instance de mailhost pour la getsion de la queue de messages, ce n'est # pas correct, il faut qu'il garantisse qu'il n'y a qu'une thread par # répertoire réel de queue. from Products.MailHost import MailHost as mh if not hasattr(mh.MailBase, '_cfenet_patched_queueThreadAlive'): mh.MailBase._cfenet_patched_queueThreadAlive = mh.MailBase.queueThreadAlive def queueThreadAlive(self): """return True/False is queue thread is working""" from Products.MailHost import MailHost as mh from os.path import realpath # Il est important d'avoir un mailer unique par répertoire th = mh.queue_threads.get(realpath(self.smtp_queue_directory)) return th and th.isAlive() mh.MailBase.queueThreadAlive = queueThreadAlive if not hasattr(mh.MailBase, '_cfenet_patched__stopQueueProcessorThread'): mh.MailBase._cfenet_patched__stopQueueProcessorThread = \ mh.MailBase._stopQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _stopQueueProcessorThread(self): """ Stop thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if mh.queue_threads.has_key(path): th = mh.queue_threads[path] th.stop() while th.isAlive(): # wait until thread is really dead mh.time.sleep(0.3) del mh.queue_threads[path] mh.LOG.info('Thread for %s stopped' % path) mh.MailBase._stopQueueProcessorThread = _stopQueueProcessorThread if not hasattr(mh.MailBase, '_cfenet_patched__startQueueProcessorThread'): mh.MailBase._cfenet_patched__startQueueProcessorThread = \ mh.MailBase._startQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _startQueueProcessorThread(self): """ Start thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if not mh.queue_threads.has_key(path): th = mh.QueueProcessorThread() th.setMailer(self._makeMailer()) th.setQueuePath(self.smtp_queue_directory) th.start() mh.queue_threads[path] = th mh.LOG.info('Thread for %s started' % path) mh.MailBase._startQueueProcessorThread = _startQueueProcessorThread -- _____________ Maric Michaud
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maric Michaud wrote:
Hi all,
I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary.
It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once.
Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails.
It looks like a bug to me: could you please report it to the tracker at: http://bugs.launchpad.net/zope2 Also, if you could convert your monkeypatch into a real diff against the source, that would help expedite. The basic procedure is outlined at: http://docs.zope.org/developer/noncommitter-svn.html E.g., get a read-only checkout of the 2.11 branch: $ cd /tmp $ svn co svn://svn.zope.org/repos/main/Zope/branches/2.11 Zope-2.11 build it and make an instance: $ cd /tmp/Zope-2.11 $ ./configure --with-python=/path/to/python2.4 && make inplace $ bin/mkzopeinstance /tmp/test_mailhost apply your changes in that tree, test them, and then create a diff using SVN: $ cd /tmp/Zope-2.11 $ svn diff -u > /tmp/mailhost_thread_fix.patch and attach that file to your bugreport. Bonus points (i.e., a faster resolution) if you can generate a unit test which fails without your changes, but passes with them. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkva1BIACgkQ+gerLs4ltQ4sPQCgv7OmVZ3ynVInCddns4T1maar UnUAoMD1CKuMS0FKkZkX+34HWq02nkTq =5qyA -----END PGP SIGNATURE-----
Tres Seaver a écrit :
I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary.
It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once.
Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails.
It looks like a bug to me: could you please report it to the tracker at:
http://bugs.launchpad.net/zope2
Also, if you could convert your monkeypatch into a real diff against the source, that would help expedite. The basic procedure is outlined at:
Ok, I'll do that, soon. -- _____________ Maric Michaud
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maric Michaud wrote:
Tres Seaver a écrit :
I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary.
It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once.
Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails.
It looks like a bug to me: could you please report it to the tracker at:
http://bugs.launchpad.net/zope2
Also, if you could convert your monkeypatch into a real diff against the source, that would help expedite. The basic procedure is outlined at:
Ok, I'll do that, soon.
Thanks for posting the bug and fix to Launchpad -- I have merged it for the next 2.11 and 2.12 releases, as well as to the trunk. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvgJEgACgkQ+gerLs4ltQ4JEgCfSRZGizktQxo6pberCM/X5PNz 1bgAoKeRw/bvooXZCQhHNxAKuM9fDUG7 =3V6A -----END PGP SIGNATURE-----
participants (2)
-
Maric Michaud -
Tres Seaver