[Zope-dev] MailHost Improvements

[Kapil Thangavelu]

There has been intermittent talk of improving mailhost's ability to
scale. i just wanted to solicit some comments to some ideas for
improving mail host performance. 

Stage I. Persistent Socket
1. make the smtp connection persistent
2. preface sending mail with a 'noop' operation to verify connection
status
3. overide __setstate__ to open connection

i'm not sure if this will yield any real world benefit but i like the
style better than opening a new connection for every new message.

Stage II. Transaction Aware
1. Store sent mail in a volatile 
2. on transaction commit begin (tpc_begin), send all mail, if error
abort transaction
3. reset volatile

again i'm not sure of real world performance, but the design seems
better and the added benefit that it will play nicely with the
transaction machinery and not send unesc. emails.


Stage III. Intelligent Q. aka the BulkMailHost

1. implement mailer thread / sole function to send mails from queue and
intelligently manage a outgoing queue and deferred queue.
2. implement extensions to sendmail tag for immediate sending of mail
which will wake/signal mailer thread after depositing into outgoing q.
extension options are

3. mailer thread on a timer, so it will wake on either a sendmail tag
with immeadiate extension or on timer expiration and send mail from
queue, messages with error due to connection or placed in deferred
queue. messages with undelivarable errors are either deleted or placed
in a deleted queue.

4. all queues are viewable and manageable from the management interface. 


obviously this makes the sendmail tag async to the message being sent.

i'm not sure if this something that really should be implemented in a
ZODB default file storage due to the high write nature of the system or
just dumped to the fs. again this seems questionable also because a
proper mta should be operating on a store and send principle
(qmail/postfix).

comments?

hmmm... looking through the source of the 2.2.2 mailhost i see a
scheduledSend method, anyone know what this is from/for?

Cheers

Kapil