Hi, Has anyone had any experience with storing a large list (several thousand) of email addresses in an object then sending an email to every one in the list through a MailHost? Will I need to send on email to all these addresses (multiple recipients), or one email for each person on the list. I am probably going to end up doing this totally within Zope since it is an easy solution to a problem, but am wondering whether I would be better off trying to get all the mail address into mailman or something. Terry. -- ----------------------------- Terry Kerr (terry@adroit.net) Adroit Internet Solutions http://www.adroit.net/ 03 9563 4461 0414 708 124
Terry Kerr wrote:
Has anyone had any experience with storing a large list (several thousand) of email addresses in an object then sending an email to every one in the list through a MailHost? Will I need to send on email to all these addresses (multiple recipients), or one email for each person on the list.
I'm not sure how well MailHost scales, but you can certainly store thousands of names in a ZODB-managed list. Tests done for the BTreeFolder product prove it.
I am probably going to end up doing this totally within Zope since it is an easy solution to a problem, but am wondering whether I would be better off trying to get all the mail address into mailman or something.
It's up to you. Mailman certainly has more mileage in this area. Shane
Shane Hathaway wrote:
I'm not sure how well MailHost scales, but you can certainly store thousands of names in a ZODB-managed list. Tests done for the BTreeFolder product prove it.
Sadly, I think MailHost doesn't scale too well :-( What it'd need to do if you're sending the mail to a few thousand people is to be non-blocking. IIRC, currently, a method/page with a <dtml-in addresses><sendmail></dtml-in> tag pair in it will only return when the mail has been sent to all the recipients. So, if the mails going to lots of people, the page takes ages to return. This is the problem I had where our mail server was taking lots of time to send messages (it was doing lots of stuff to make sure the address was valid) so submitting posts on Squishdot.org was taking ages too. What, IMHO, is really needed is a mailhost/sendmail tag type thing that gets a message and a list of addresses to send it to. If it could do that in a seperate thread/process/whatever so that whatever calls it doesn't block, that'd be great. Of course, it'd need to have a 'hook back' method provided so any errors that occured could be dealt with. I wonder how mailman does this stuff and if the code could be borrowed for Zope? cheers, Chris
On Tue, 3 Oct 2000, Chris Withers wrote:
What, IMHO, is really needed is a mailhost/sendmail tag type thing that gets a message and a list of addresses to send it to. If it could do that in a seperate thread/process/whatever so that whatever calls it doesn't block, that'd be great. Of course, it'd need to have a 'hook back' method provided so any errors that occured could be dealt with.
I wonder how mailman does this stuff and if the code could be borrowed for Zope?
I don't have a quick answer. (It's been a long time since i worked on mailman, and this stuff in particular has changed a lot.) It looks like the code in Mailman/Handlers/SMTPDirect.py is what's of interest, and it doesn't depend too elaborately on the rest of mailman, which is good. I'm have no idea whether it solves the problem of doing unblocking delivery in a way that lets the delivery requester get a status back. Ken klm@digicool.com
Chris Withers wrote:
Shane Hathaway wrote:
I'm not sure how well MailHost scales, but you can certainly store thousands of names in a ZODB-managed list. Tests done for the BTreeFolder product prove it.
Sadly, I think MailHost doesn't scale too well :-(
What it'd need to do if you're sending the mail to a few thousand people is to be non-blocking. IIRC, currently, a method/page with a <dtml-in addresses><sendmail></dtml-in> tag pair in it will only return when the mail has been sent to all the recipients. So, if the mails going to lots of people, the page takes ages to return.
Perhaps it would be better to solve this by configuring your sendmail (or other SMTP MTA) to be non-blocking, i.e. store-and-forward. Also you could send just one message to all the recipients instead of sending individual messages.
This is the problem I had where our mail server was taking lots of time to send messages (it was doing lots of stuff to make sure the address was valid) so submitting posts on Squishdot.org was taking ages too.
What, IMHO, is really needed is a mailhost/sendmail tag type thing that gets a message and a list of addresses to send it to. If it could do that in a seperate thread/process/whatever so that whatever calls it doesn't block, that'd be great. Of course, it'd need to have a 'hook back' method provided so any errors that occured could be dealt with.
I wonder how mailman does this stuff and if the code could be borrowed for Zope?
Maybe you could just use mailman as MTA, greating temporary mailing-lists. -------- Hannu
Hannu Krosing wrote:
Perhaps it would be better to solve this by configuring your sendmail (or other SMTP MTA) to be non-blocking, i.e. store-and-forward.
Can you tell me how to set that up with exim?
Also you could send just one message to all the recipients instead of sending individual messages.
Urm, they could be pretty unfriendly mail messages if they're being sent to a coupla thousand people. The bandwidth dcoming out of your server alone would be far higher than it needed to be :-(
I wonder how mailman does this stuff and if the code could be borrowed for Zope?
Maybe you could just use mailman as MTA, greating temporary mailing-lists.
Any how-to's for this or how hard it would be to set up? cheers, Chris
Techniques for effeciently sending a message to a large number of recipients are covered extensively in majordomo documentation - www.greatcircle.com/majordomo. Also check out ftp://cs.utk.edu/pub/moore/bulk_mailer Bill. On Wed, 4 Oct 2000, Chris Withers wrote:
Also you could send just one message to all the recipients instead of sending individual messages.
Urm, they could be pretty unfriendly mail messages if they're being sent to a coupla thousand people. The bandwidth dcoming out of your server alone would be far higher than it needed to be :-(
Chris Withers wrote:
Hannu Krosing wrote:
Perhaps it would be better to solve this by configuring your sendmail (or other SMTP MTA) to be non-blocking, i.e. store-and-forward.
Can you tell me how to set that up with exim?
Also you could send just one message to all the recipients instead of sending individual messages.
Urm, they could be pretty unfriendly mail messages if they're being sent to a coupla thousand people. The bandwidth dcoming out of your server alone would be far higher than it needed to be :-(
How would the bandwidth change??
I wonder how mailman does this stuff and if the code could be borrowed for Zope?
Maybe you could just use mailman as MTA, greating temporary mailing-lists.
Any how-to's for this or how hard it would be to set up?
cheers,
Chris
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- ----------------------------- Terry Kerr (terry@adroit.net) Adroit Internet Solutions http://www.adroit.net/ 03 9563 4461 0414 708 124
Terry Kerr wrote:
Chris Withers wrote:
Hannu Krosing wrote:
Also you could send just one message to all the recipients instead of sending individual messages.
Urm, they could be pretty unfriendly mail messages if they're being sent to a coupla thousand people. The bandwidth dcoming out of your server alone would be far higher than it needed to be :-(
How would the bandwidth change??
Well, suppose you send a message of, say, 2 KByte (about one page of typed text) to 1,000 recipients. You'll make about 1,000 connections to the recipients' MX hosts. Each of these will be around 42 KByte large. Huh? Why that? you ask. Because in the header of each mail, the whole recipient list will be listed under To: or Cc: (I assumed 40 bytes per address as a roundabout figure). In the envelope (what the SMTP protocol *uses* to deliver the message, only one address will be used, the body To: list is mainly for the recipients' use (to show them who else got it). Besides, there are other problems why you will want separate mails: - privacy: You would not like to have "your" recipients end up on other people's mailing lists - stability: When E-mail bounces, you would like to hanle it intelligently, like trying this address once again (as it could be a temporary outage) and take the address off the list if it still bounces a week or so later - stability again: You woulld not like to be blacklisted as either a spammer or clueless by the recipients' postmaster :-) I would probably use a stable MTA (like qmail, which has good mailing list processing software, or postfix) for the job. Cheers, Jan
"Jan H. Haul" wrote:
Each of these will be around 42 KByte large.
Huh? Why that? you ask. Because in the header of each mail, the whole recipient list will be listed under To: or Cc: [snip] - privacy: You would not like to have "your" recipients end up on other people's mailing lists
What about using BCC: ? Michael Bernstein.
Michael Bernstein wrote:
"Jan H. Haul" wrote:
Each of these will be around 42 KByte large.
Huh? Why that? you ask. Because in the header of each mail, the whole recipient list will be listed under To: or Cc: [snip] - privacy: You would not like to have "your" recipients end up on other people's mailing lists
What about using BCC: ?
That would effectively sending one separate mail to each recipient, without giving you the full benefit of being able to identify each in case of bounces. I'd recommend using mailing list processing software (ezmlm on qmail is especially handy, or majordomo) for the task - it is in fact written for just this, and makes your life so much easier. If someone insists in reinventing wheels, well, Monty Python has a nice sketch on the topic :-) Jan
On Wed, 4 Oct 2000, Hannu Krosing wrote:
Perhaps it would be better to solve this by configuring your sendmail (or other SMTP MTA) to be non-blocking, i.e. store-and-forward.
This seems like a good idea. It makes it hard to get disposition status for the delivery in the transaction - but that problem is intrinsic to email communications, and probably best addressed by using the "sender" header (i think it is), so delivery failure notifications go to someone who needs to know about the failure. Even better - but lots more complicated: have the failure notifications directed to a program which queues the fact of the failure for the sending program to use. I haven't used it, but contemporary MTAs have catchall aliases, by which you can handle a pattern of recipients. Have the envelope sender fit the pattern, and encode the information about the program that needs to get back the failure information. I think this kind of architecture is necessary for comprehensive mail sending from zope, with delivery failure accounting, since mail sending is inherently asynchronous.
Maybe you could just use mailman as MTA, greating temporary mailing-lists.
This is also a good idea in principle, but would wind up being cumbersome. Mailling list "objects" are heavy weight components in mailman - each one having a list metadata and archive directory, mta aliases, etc. Plus, you may have to leave the list up until the last delivery is completed (successful or not). Better, i think, to adopt techniques and steal code from mailman, to take the pieces you want. Ultimately, i don't think you'd be taking that much - there's a lot in python that gets you a lot of the way. It's the embedded knowledge that's useful... -- Ken klm@digicool.com
participants (8)
-
Bill Welch -
Chris Withers -
Hannu Krosing -
Jan H. Haul -
Ken Manheimer -
Michael Bernstein -
Shane Hathaway -
Terry Kerr