[Zope-dev] MailHost Improvements

[Alec Mitchell]

On Wed, Aug 12, 2009 at 9:42 PM, Andreas Jung<lists at zopyx.com> wrote:
> On 13.08.09 01:03, Alec Mitchell wrote:
>> Hello,
>>
>> I've been working on making Plone use the standard Zope MailHost  in
>> place of the custom Products.SecureMailHost we've been using since
>> Plone 2.1 (See: http://dev.plone.org/plone/ticket/8814).  During this
>> process I've run into a couple bugs in the MailHost implementation and
>> I believe it is missing some essential functionality.
>>
>> The most significant issue is that if you call send() with a
>> messageText containing just the message body, and that body has a ':'
>> in it (e.g. a url) the body will be treated as a header and you'll
>> send a nonsense message.  The current implementation of send() also
>> puts a fairly large burden on developers who want to generate simple,
>> correctly encoded messages.  Finally, send() relies heavily on the
>> long deprecated 'rfc822' and 'mimetools' modules which have been
>> removed from Python 3.0.
>>
>> I've attached a patch that updates MailHost to use the 'email' module
>> for parsing and generating messages.  In addition to fixing the issues
>> that I ran across, and maintaining compatibility, it provides a number
>> of new features:
>>
>> * send and sendTemplate accept an optional charset argument.  Using
>> this will set the content-type charset, as well as trigger appropriate
>> encoding if needed.
>>
>> * send and sendTemplate accept an optional msg_type argument which
>> will set the content type header for the message.
>>
>> * The messageText, mfrom, mto, and subject arguments may now be
>> unicode or encoded non-ascii strings, provided a charset is given.
>> Any unicode input will be automatically encoded to the provided
>> charset (or the default charset).  Headers will be further encoded in
>> compliance with rfc2822.  The message body will be further encoded
>> using a transfer encoding determined by the email.Charset registry
>> (e.g. 7bit for us-ascii, quoted-printable for utf-8 and iso8859,
>> base64 for most other encodings).
>>
>> * The messageText argument now accepts email.Message.Message objects
>> directly.
>>
>> I'm attaching a patch that includes these changes as well as tests for
>> all new functionality.  I hope to integrate these changes into Zope
>> 2.12 before final release, but would like to hear the opinions of Zope
>> developers before committing.  Though these are fairly significant
>> changes, I believe they provide very useful functionality as well as
>> at least one critical fix, while maintaining 100% compatibility.
>
> This comes very, very late. We are pretty close to a release. Please put
> the changes on the trunk only.
> We will check after my vacation if we can move it into the 2.12 beta.

I've put my latest changes on Zope trunk.  All the existing tests pass
(with a couple essentially cosmetic modifications in the MailHost
tests), and there are 14 new tests which verify both existing and new
functionality, as well as the fixed bugs.  The new behavior should be
identical to the existing behavior when the new charset and msg_type
parameters aren't used, with the following exceptions:

1) Passing a message body containing a ':' no longer produces a nonsense email.
2) Providing unicode strings for the text or headers no longer results
in a garbage message (it may produce a UnicodeEncodeError though).
3) 8bit (encoded) strings provided as headers will be converted to
7bit, using encoding determined in messageText headers or the default
encoding.

It would be very helpful to have these changes in Zope 2.12;
otherwise, Plone 4.0 will be stuck with our unmaintained
SecureMailHost product for yet another release in order to provide
equivalent functionality.  Moving to a standard Zope MailHost would be
a big benefit for Plone, and all Zope users will benefit from the
ability to easily send properly formed non-ASCII messages.

Alec

P.S. Andreas, if there's an address where I can send some nice wine to
help facilitate the merge into 2.12, let me know ;-)