zope.sendmail and critical transaction errors.
With Zope2's MailHost now using zope.sendmail, we're seeing some critical errors when sending mail when the mail server domain name is misconfigured. http://dev.plone.org/plone/ticket/10675 (these are triggered by a password reset mail, the registration mail is sent immediately). This is because zope.sendmail.delivery.MailDataManager sends mail in tpc_finish when using DirectMailDelivery. While MailDataManager makes sense for QueuedMailDelivery (msg.commit should never fail) for DirectMailDelivery it seems wrong. To fix this, DirectMailDelivery should use a commit hook - there are two options: * After Commit Hook Ensures mail is only sent once in event of a request being retried, but errors are swallowed so no feedback that there is a problem to the browser. * Before Commit Hook Mail me be sent multiple times in event of a request being retried due to a conflict error, but errors propagate to the browser. I think the Before Commit Hook option is probably best here. DirectMailDelivery should only be used for testing anyway, or at least only on very small sites in production - QueuedMailDelivery will scale better. For Zope 2.12 / Plone 4.0 we have the additional problem that Zope 2.12 is incompatible with zope.sendmail 3.7.x / trunk due to a zope.component 3.8 dependency. I think this issue is serious enough to warrant backporting this fix to the zope.sendmail 3.6.x branch. Patches attached for comment. Laurence
Hi. On Wed, Jun 23, 2010 at 1:59 PM, Laurence Rowe <l@lrowe.co.uk> wrote:
I think the Before Commit Hook option is probably best here. DirectMailDelivery should only be used for testing anyway, or at least only on very small sites in production - QueuedMailDelivery will scale better.
A before commit hook sounds like the right thing to do indeed.
For Zope 2.12 / Plone 4.0 we have the additional problem that Zope 2.12 is incompatible with zope.sendmail 3.7.x / trunk due to a zope.component 3.8 dependency. I think this issue is serious enough to warrant backporting this fix to the zope.sendmail 3.6.x branch.
Feel free to backport this to zope.sendmail 3.5.x (3.6.0 introduced the backwards incompatible zope.component version). I can cut a new release and include it in the next 2.12.x release. Hanno
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Laurence Rowe wrote:
With Zope2's MailHost now using zope.sendmail, we're seeing some critical errors when sending mail when the mail server domain name is misconfigured. http://dev.plone.org/plone/ticket/10675 (these are triggered by a password reset mail, the registration mail is sent immediately).
This is because zope.sendmail.delivery.MailDataManager sends mail in tpc_finish when using DirectMailDelivery. While MailDataManager makes sense for QueuedMailDelivery (msg.commit should never fail) for DirectMailDelivery it seems wrong.
To fix this, DirectMailDelivery should use a commit hook - there are two options:
* After Commit Hook
Ensures mail is only sent once in event of a request being retried, but errors are swallowed so no feedback that there is a problem to the browser.
* Before Commit Hook
Mail me be sent multiple times in event of a request being retried due to a conflict error, but errors propagate to the browser.
I think the Before Commit Hook option is probably best here. DirectMailDelivery should only be used for testing anyway, or at least only on very small sites in production - QueuedMailDelivery will scale better.
For Zope 2.12 / Plone 4.0 we have the additional problem that Zope 2.12 is incompatible with zope.sendmail 3.7.x / trunk due to a zope.component 3.8 dependency. I think this issue is serious enough to warrant backporting this fix to the zope.sendmail 3.6.x branch.
Patches attached for comment.
- -sys.maxint to changing the mail delivery to a non-transactional form. Switching to a before commit hook is the *worst* of the optoins: it defeats the whole point of using zope.sendmail in the first place, which was to avoidi multiple e-mails on retried requests: if the application doesn't want the mail to be transactional, then it should just send it immediately (which is what the MailHost used to do). At that point, the user would get an error screen, which couold only be cleared by the admin fixing the mail configuration. The misconfiguration of the mail host in transactional mode *does* cause the server to go into a "lockdown" mode (because errors are not supposed to be raised in 'tpc_finish'). There is *no* database corruption, but the transaction manager can't know that, and so it deliberately refuses to commit more transactions until the admin un-futzes the problem (in this case, by fixing the mailhost configuration) and restarts the server. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwiJk4ACgkQ+gerLs4ltQ4VnQCeNdJCHQojfrBQymejpM/WT7YN s+EAn1XY2ju5B1taLMax8cx4BUBHcR3A =cgaT -----END PGP SIGNATURE-----
Tres Seaver wrote:
Patches attached for comment.
- -sys.maxint to changing the mail delivery to a non-transactional form. Switching to a before commit hook is the *worst* of the optoins: it defeats the whole point of using zope.sendmail in the first place, which was to avoidi multiple e-mails on retried requests: if the application doesn't want the mail to be transactional, then it should just send it immediately (which is what the MailHost used to do). At that point, the user would get an error screen, which couold only be cleared by the admin fixing the mail configuration.
The misconfiguration of the mail host in transactional mode *does* cause the server to go into a "lockdown" mode (because errors are not supposed to be raised in 'tpc_finish'). There is *no* database corruption, but the transaction manager can't know that, and so it deliberately refuses to commit more transactions until the admin un-futzes the problem (in this case, by fixing the mailhost configuration) and restarts the server.
Yeah, what he said... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 6/24/10 09:30 , Chris Withers wrote:
Tres Seaver wrote:
The misconfiguration of the mail host in transactional mode *does* cause the server to go into a "lockdown" mode (because errors are not supposed to be raised in 'tpc_finish'). There is *no* database corruption, but the transaction manager can't know that, and so it deliberately refuses to commit more transactions until the admin un-futzes the problem (in this case, by fixing the mailhost configuration) and restarts the server.
Yeah, what he said...
It's not as easy as that, though. Mailhost misconfiguration is just one reason why errors may be raised during the actual sending. The receiver address may be wrong, which is where I tend to see it. This is not a condition I could reasonably test for in my code before attempting to send though the MailHost. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkwjDWUACgkQRAx5nvEhZLKf9ACgjfp7Nxf5B+2BwT6Egp6mhuVV yGIAoKhuD8sEGT+mn6NcbTvqlWxduYsi =LIL8 -----END PGP SIGNATURE-----
On 6/24/10 09:46 , Jens Vagelpohl wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 6/24/10 09:30 , Chris Withers wrote:
Tres Seaver wrote:
The misconfiguration of the mail host in transactional mode *does* cause the server to go into a "lockdown" mode (because errors are not supposed to be raised in 'tpc_finish'). There is *no* database corruption, but the transaction manager can't know that, and so it deliberately refuses to commit more transactions until the admin un-futzes the problem (in this case, by fixing the mailhost configuration) and restarts the server.
Yeah, what he said...
It's not as easy as that, though. Mailhost misconfiguration is just one reason why errors may be raised during the actual sending. The receiver address may be wrong, which is where I tend to see it. This is not a condition I could reasonably test for in my code before attempting to send though the MailHost.
What might be interesting is a MailHost for non-bulk mail sending which does the SMTP HELO/MAIL FROM/RCPT TO bits immediately and only delays the DATA command until transaction commit time. That will allow you to detect almost all SMTP configuration problems and address errors in your code while still getting transaction support. Personally I've updated all my code to sending mail directly and not using zope.sendmail. Being able to catch and handle errors is much more useful to me than preventing extremely rare duplicate emails. Wichert.
On 2010-06-23, at 1359, Laurence Rowe wrote:
I think the Before Commit Hook option is probably best here. DirectMailDelivery should only be used for testing anyway, or at least only on very small sites in production - QueuedMailDelivery will scale better.
Sorry, I'm somewhat late to this thread, in the process of moving to Germany and missed it. I've committed some partial fixes that don't rely on using transaction hooks: - Products.MailHost now no longer duplicates code in zope.sendmail - zope.sendmail mailers grow a vote() method to allow them to veto transactions if they know they'll fail. Currently SMTPMailer will go as far as HELO before accepting the vote. - All exceptions in tpc_finish are redirected to the log instead of being raised This is similar to what Wichert described, the upshot is that missing and non-connectable servers cause a transaction abort but deeper problems can fail part-way through sending the emails (as currently happens) but without breaking the ZODB contracts. It has been suggested on IRC that we should also have an event fire on failed mail sending so applications can display a message in the UI, which seems like a good idea. This obviously isn't perfect but it retains the basic functionality of sendmail as it is currently. Matt
participants (7)
-
Chris Withers -
Hanno Schlichting -
Jens Vagelpohl -
Laurence Rowe -
Matthew Wilkes -
Tres Seaver -
Wichert Akkerman