Proposal for mail-in to Zope
Hi, We (NIP - in the form of Chris Withers, Stephen Harrison, and Simon Coles) have put some thought into the question of how to get emails into Zope, and have come up with a design which we thought we'd pass by the list before we start doing something. The problem we are trying to solve is basically being to email "something@myserver.me.com" and that email ends up in the Zope ZODB processed in whatever way is appropriate. We aren't worried about sending email, that's done well at the moment. We also wanted to keep things as simple and quick as possible :-) This means trying to avoid getting involved with mail systems as much as we can. Most MTAs can be setup to pass an email to the stdin of a program. Sendmail will do this, and Exim (http://www.exim.org/) will also pay attention to what that program returns and queue the message for re-try later if it fails. So using Exim, we don't have to get into any messy stuff about queuing mails if the Zope server is down. Some alternatives we considered and didn't go for: - write something in Zope to listen for SMTP connections, effectively large portions of an MTA. This would be cool but painful. - pull mail from a POP or IMAP server. This had the downside that it introduced polling into the system (slow) and also required something to happen on a schedule, which doesn't happen in Zope yet. So: - Within Exim, use the "pipe" transport to get email for a particular host sent to a program on the standard input. If the program called returns certain codes then Exim will queue the message and try again. - This program takes the email message and puts it into Zope, probably by calling a DTML Method or something. This would probably be configured by objects in the Zope ZODB which say effectively "When you get email for this address, then call this Method". - The DTML Method is responsible for doing the right things to get the email message into the ZODB in whatever form is appropriate. We haven't yet figured out how to make sure the above mail handling program can find all the relevant configuration documents. Is there some way of efficiently finding all instances of a particular ZClass? (Chris W. still isn't sure that's how the configuration will work but that's the plan for the moment, so any thoughts would be appreciated) What we'll do is a first version that hard codes everything in the above script and prove the principle of the thing, and then move onto the configuration. Does this sound like what people need? Any thoughts, comments, warnings? I don't know yet what kind of schedule we'll be able to do this on but drop me a mail and when there's something to play with we'll let you know. Simon -- --------- My opinions are my own, NIP's opinions are theirs ---------- Simon J. Coles Email: simon@nipltd.com New Information Paradigms Work Phone: +44 1344 753703 http://www.nipltd.com/ Work Fax: +44 1344 753742 =============== Life is too precious to take seriously ===============
----- Original Message ----- From: "Simon Coles" <simon@nipltd.com> To: <zope@zope.org> Sent: Wednesday, May 31, 2000 11:54 AM Subject: [Zope] Proposal for mail-in to Zope
So: - Within Exim, use the "pipe" transport to get email for a particular host sent to a program on the standard input. If the program called returns certain codes then Exim will queue the message and try again.
- This program takes the email message and puts it into Zope, probably by calling a DTML Method or something. This would probably be configured by objects in the Zope ZODB which say effectively "When you get email for this address, then call this Method".
Hmm... "takes the email message and puts it into Zope". Using ZClient or XML-RPC? ZClient is probably the easiest way to go. You'd probably want a simple script configured in the file system to send the message on to method in Zope via ZClient. That method will be part of a MailHandler object (or some such) that can have rules based on the email address that the message is going to. These rules will likely pass the message on to another method (probably a PythonMethod, since this is programming we're talking about). It might be nice if the MailHandler broke the message up into usable fields instead of a plain text stream. I think there's a python module for doing that. Kevin
I think this is pretty much what we're intending to do. We haven't decided about XML-RPC Vs ZClient. The aim is to get something (anything) working, however hard coded, and then improve it. Simon "Kevin Dangoor" <kid@kendermedia.com> wrote:
Hmm... "takes the email message and puts it into Zope". Using ZClient or XML-RPC? ZClient is probably the easiest way to go. You'd probably want a simple script configured in the file system to send the message on to method in Zope via ZClient. That method will be part of a MailHandler object (or some such) that can have rules based on the email address that the message is going to. These rules will likely pass the message on to another method (probably a PythonMethod, since this is programming we're talking about). It might be nice if the MailHandler broke the message up into usable fields instead of a plain text stream. I think there's a python module for doing that.
Kevin
-- --------- My opinions are my own, NIP's opinions are theirs ---------- Simon J. Coles Email: simon@nipltd.com New Information Paradigms Work Phone: +44 1344 753703 http://www.nipltd.com/ Work Fax: +44 1344 753742 =============== Life is too precious to take seriously ===============
Simon Coles wrote:
I think this is pretty much what we're intending to do. We haven't decided about XML-RPC Vs ZClient. The aim is to get something (anything) working, however hard coded, and then improve it.
Use XML-RPC - it's a lot more powerful (e.g. it returns lists and dictionaries and integers and floats, while ZClient can only return strings.) -- Itamar S.T. itamar@maxnm.com "It don't get thingier than that!"
On Wed, May 31, 2000 at 04:54:24PM +0100, Simon Coles wrote:
The problem we are trying to solve is basically being to email "something@myserver.me.com" and that email ends up in the Zope ZODB processed in whatever way is appropriate.
I've been thinking about this too, reached more or less the same conclusions, and decided not to work on it because I'm already lame enought on keeping up with the hundreds of stuff I _am_ working on :-) Anyway:
Most MTAs can be setup to pass an email to the stdin of a program. Sendmail will do this, and Exim (http://www.exim.org/) will also pay attention to what that program returns and queue the message for re-try later if it fails. So using Exim, we don't have to get into any messy stuff about queuing mails if the Zope server is down.
This is probably the best solution, but don't forget virtual hosting: the method for deciding which method to call should include the whole address (not only the username) and turn up something similar to the rules used by zmailer or qmail. The problem here is that most MTAs don't pass the whole envelope address to such a "piped" program. As zmailer is mostly scripted, I figured it would probably be the best/easier solution, check it out. (Plus, it starts with Z ;-) hehehe)
Some alternatives we considered and didn't go for: - write something in Zope to listen for SMTP connections, effectively large portions of an MTA. This would be cool but painful.
Agreed on both cool and painful. Perhaps the best solution in the long run, but we need a ZODB more friendly to high-writing first, I think.
- pull mail from a POP or IMAP server. This had the downside that it introduced polling into the system (slow) and also required something to happen on a schedule, which doesn't happen in Zope yet.
I don't see advantages to this approach.
- This program takes the email message and puts it into Zope, probably by calling a DTML Method or something. This would probably be configured by objects in the Zope ZODB which say effectively "When you get email for this address, then call this Method".
You don't even have to know what kind of method it is, and in theory it doesn't have to be in the same server even. For each mail address (or domain, in case of fallbacks, if you want to implement that) you assign an URL, username and perhaps password, and submit the message via POST somehow. This tool would end up so easy to write (in principle) and generic that it wouldn't even be tied to Zope. (Yes, you could use ZPublisher.Client or XML-RPC or something more Zope-specific - but why? What would you gain by that?)
We haven't yet figured out how to make sure the above mail handling program can find all the relevant configuration documents. Is there some way of efficiently finding all instances of a particular ZClass?
There is a Design Pattern covering this. Essentially, the class registers all instances somewhere when they're built - you can do that in the constructor. But there are two different things here... either you store the configuration outside the ZODB and send to a different method depending on this configuration, _or_ you send everything to a single method and use a configuration in the ZODB to forward the message to the correct handling method. Mixing the approachs is not a good idea IMHO. []s, |alo +---- -- Hack and Roll ( http://www.hackandroll.org ) News for, uh, whatever it is that we are. http://zope.gf.com.br/lalo mailto:lalo@hackandroll.org pgp key: http://zope.gf.com.br/lalo/pessoal/pgp Brazil of Darkness (RPG) --- http://zope.gf.com.br/BroDar
Some alternatives we considered and didn't go for: - write something in Zope to listen for SMTP connections, effectively large portions of an MTA. This would be cool but painful.
Providing a basic SMTP server is not at all difficult to do in Python. I did a cut-down one once as part of a test system, and extending it would be straightforward (the cut-downness was not in structure, just did not bother to implement the SMTP commands I did not need - the underlying design was a state-machine [natch!] - however one problem I did come across was that it was not possible to implement the state machine completely from the RFC, which contains ambiguities). Some potential troublespots in re-visiting this work: - error handling, if done properly, might cause some problems (the state machine helps a lot here, but I cannot guarantee that I did not miss something obvious) - emails nowadays can be very large, with MIME encoded movies and stuff. Each received email has to be stored somewhere - can we (would we want to?) reliably store this kind of data in the ZODB?
- pull mail from a POP or IMAP server. This had the downside that it introduced polling into the system (slow) and also required something to happen on a schedule, which doesn't happen in Zope yet.
Funny you should mention this. I am considering moving my domain from the current ISP to a friend's server. One of my current ISP's services that I use is webmail, so I was looking at how this could be done after the move. Sending mail through a web interface is a doddle, but what about viewing received emails? I had got as far as realizing that something could be built on Python's support for POP3 - use POP3 to collect the headers of currently available messages, create a web page displaying them, with each entry linked to a page that would download (without deleting) an individual email. I think that my current ISP's webmail system must be doing something like this. So far it looks fairly straightforward to implement. Richard (who now realizes that he has to find that old Python SMTP server code - now I wonder where I put it ...)
participants (5)
-
Itamar Shtull-Trauring -
Kevin Dangoor -
Lalo Martins -
Richard Folwell -
Simon Coles