Zope (or Python?) not completely thread-safe...?!
hi! i spent several hours looking for a bug which caused my whole zope system to be hanging. the solution was this: i've been using <dtml-sendmail> to send out emails but due to a network change my smtp-host had changed and i was trying to use a server that wasn't answering to my smtp-request due to some firewall-something. ok, one zope thread was waiting for a reply from the smtp-server but as a consequence all other zope threads were blocked in some way so that i couldn't access my zope system anymore. is that a normal behaviour...?! OS is linux-2.2 ... i would have expected one thread to be waiting and the others to be serving requests as usual... thilo -- mezger@innominate.de innominate AG networking people fon: +49.30.308806-11 fax: -77 web: http://innominate.de pgp: /pgp/tm
Thilo Mezger wrote:
hi!
i spent several hours looking for a bug which caused my whole zope system to be hanging. the solution was this:
i've been using <dtml-sendmail> to send out emails but due to a network change my smtp-host had changed and i was trying to use a server that wasn't answering to my smtp-request due to some firewall-something.
ok, one zope thread was waiting for a reply from the smtp-server but as a consequence all other zope threads were blocked in some way so that i couldn't access my zope system anymore.
is that a normal behaviour...?! OS is linux-2.2 ...
Not in general. Apparently the smtp code is making some call that is blocking the interpreter.
i would have expected one thread to be waiting and the others to be serving requests as usual...
Me to. In general, that's the case. Someone should dig into the SMTP code to see where the interpreter is being blocked. FWIW, I'd like to see SMTP work quite a bit differently. First, I'd like to see SMTP sending delayed until transaction commit. Second, I'd like to see sending done asynchronously. It would be nice, when using ZServer, to take advantage of asyncore to send email messages. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jim Fulton wrote:
ok, one zope thread was waiting for a reply from the smtp-server but as a consequence all other zope threads were blocked in some way so that i couldn't access my zope system anymore.
is that a normal behaviour...?! OS is linux-2.2 ...
Not in general. Apparently the smtp code is making some call that is blocking the interpreter.
From some postings on this list I got an impression that the code for sendmail tag is not using smtplib, but does everything itself.
Are there some plans to start using pythons smtplib instead of your own code ? This would avoid duplicating effort (and IIRC fix some problems with Exchange server).
i would have expected one thread to be waiting and the others to be serving requests as usual...
Me to. In general, that's the case. Someone should dig into the SMTP code to see where the interpreter is being blocked.
Or just use the patches announced on this list a while ago and test if the problem persists.
FWIW, I'd like to see SMTP work quite a bit differently. First, I'd like to see SMTP sending delayed until transaction commit.
An option to have all tracebacks posted to some address would be also nice. Probably can be achieved by embeddind <sendmail ...> in standard_error_message ?
Second, I'd like to see sending done asynchronously. It would be nice, when using ZServer, to take advantage of asyncore to send email messages.
Could be nice, but would need almost complete rewrite and also make it impossible to share effort with main python development for whom medusa code is not very appealing. ------- Hannu
On Fri, 11 Feb 2000, Jim Fulton wrote:
i would have expected one thread to be waiting and the others to be serving requests as usual...
Me to. In general, that's the case. Someone should dig into the SMTP code to see where the interpreter is being blocked.
FWIW, I'd like to see SMTP work quite a bit differently. First, I'd like to see SMTP sending delayed until transaction commit. Second, I'd like to see sending done asynchronously. It would be nice, when using ZServer, to take advantage of asyncore to send email messages.
Hey, Jim! We are in the process of transitioning our main web asset to Zope. This site sends broadcast e-mails to all of its members (10,000+) when an important news item is posted to the site, which can potentially happen more than once a day. I am obviously (due to the volume we send out) interested in providing Zope with more robust, high performance SMTP mail capabilities. The items you mention above would be a great start to addressing our needs. Therefore, I can probably find some (in-office and home) time to work on this. I will see what I can do on my own, but I'm not too familiar with the transaction internals of Zope, or ayncore/ZServer. Any comments from you or the list on architecture, or simply tips and/or things to think about, would do much to speed my progress, I'm sure. My schedule is a bit crazy for the next few weeks, so no promises, but I will give it a shot.
Jim
--Jeff --- Jeff K. Hoffman 704.849.0731 x108 Chief Technology Officer mailto:jeff@goingv.com Going Virtual, L.L.C. http://www.goingv.com/
"Jeff K. Hoffman" wrote:
We are in the process of transitioning our main web asset to Zope. This site sends broadcast e-mails to all of its members (10,000+) when an important news item is posted to the site, which can potentially happen more than once a day.
I guess that more productive use of time would be letting the maillist expansion to sendmail or similar program and work on getting the interface to it right (and maybe writing some HowTo-s on how to set it up to always store-and-forward) Of course this will not work very well if you want to give a personalized message to each list member. Still it may be wise to just prepare the messages and let a well-tested MTA handle the actual distribution. An interesting task would be to use /usr/sbin/sendmail executable in os.system (or pipe, i.e. popen, popen2 or popen3) call to do the processing instead of talking to smtp socket.
I am obviously (due to the volume we send out) interested in providing Zope with more robust, high performance SMTP mail capabilities. The items you mention above would be a great start to addressing our needs.
Still you could end up rewriting whole sendmail, which is at least a moderate size project, even if done in python ;) ----------------- Hannu
On Fri, 11 Feb 2000, Hannu Krosing wrote:
"Jeff K. Hoffman" wrote:
We are in the process of transitioning our main web asset to Zope. This site sends broadcast e-mails to all of its members (10,000+) when an important news item is posted to the site, which can potentially happen more than once a day.
I guess that more productive use of time would be letting the maillist expansion to sendmail or similar program and work on getting the interface to it right (and maybe writing some HowTo-s on how to set it up to always store-and-forward)
I assume you mean using a Cc: or Bcc: to let the MTA distribute a single message body multiple times? Sure, for our purposes here, that is probably what we will do.
Of course this will not work very well if you want to give a personalized message to each list member. Still it may be wise to just prepare the messages and let a well-tested MTA handle the actual distribution.
We do not need a specialized message per user. The users are divided into affiliation groups, though, so we will have one e-mail go to 200 people, the next to 150 people, the next to 413 people, etc. But, this should pose no problem.
An interesting task would be to use /usr/sbin/sendmail executable in os.system (or pipe, i.e. popen, popen2 or popen3) call to do the processing instead of talking to smtp socket.
This works fine, and is the way our current (proprietary) system operates. My personal opinion, however, is that talking to an SMTP socket makes the system easier to configure, easier to install (doesn't require ANY mailer to be installed on the web server), and all around better. Of course, I could be wrong.
I am obviously (due to the volume we send out) interested in providing Zope with more robust, high performance SMTP mail capabilities. The items you mention above would be a great start to addressing our needs.
Still you could end up rewriting whole sendmail, which is at least a moderate size project, even if done in python ;)
Oh, no! I will not attempt anything close to rewriting sendmail. The changes Jim mentioned would mean: 1) A call to <dtml-sendmail> stores the message as part of the current transaction, and 2) When the transaction is committed, any e-mails which are included in the transaction are queued for asynchronous delivery to the MailHost when possible. This would allow, for instance, us to send out 500 e-mails (one of our affiliation groups) and abort all 500 of them if the enclosing transaction is rolled back (say, a database error during e-mail construction.) This would also mean that 500 e-mails could be queued for delivery VERY quickly without blocking while they are sent to the SMTP host. This is very attractive to us. We will also require some other features, such as: o Submitting an e-mail to be broadcast creates a Job. o Users with appropriate permissions can view a list of jobs, pause a job, resume a job, kill a job, etc. o Users with appropriate permissions can view job status, such as the rate that e-mails are being relayed to the MailHost, the lifetime of the job thus far, estimated time remaining, etc. I don't think achieving the above goals requires anything close to the effort of sendmail. Someone correct me if I'm wrong. ;-)
Hannu
--Jeff --- Jeff K. Hoffman 704.849.0731 x108 Chief Technology Officer mailto:jeff@goingv.com Going Virtual, L.L.C. http://www.goingv.com/
"Jeff K. Hoffman" wrote:
On Fri, 11 Feb 2000, Hannu Krosing wrote:
"Jeff K. Hoffman" wrote:
We are in the process of transitioning our main web asset to Zope. This site sends broadcast e-mails to all of its members (10,000+) when an important news item is posted to the site, which can potentially happen more than once a day.
I guess that more productive use of time would be letting the maillist expansion to sendmail or similar program and work on getting the interface to it right (and maybe writing some HowTo-s on how to set it up to always store-and-forward)
I assume you mean using a Cc: or Bcc: to let the MTA distribute a single message body multiple times? Sure, for our purposes here, that is probably what we will do.
Actually I was meaning generating an alias for the list, andt refreshing it in /etc/aliases and then sending the announcement to the alias. I am doing it for some sites and it works quite well.
An interesting task would be to use /usr/sbin/sendmail executable in os.system (or pipe, i.e. popen, popen2 or popen3) call to do the processing instead of talking to smtp socket.
This works fine, and is the way our current (proprietary) system operates. My personal opinion, however, is that talking to an SMTP socket makes the system easier to configure, easier to install (doesn't require ANY mailer to be installed on the web server), and all around better. Of course, I could be wrong.
I used to think so as well, until today one linux server started acting up: when ipchains were in use it refused to send mail a couple of times, putting NULLCONNECTION bla bla bla in logfile. THE WORST PART WAS that the sendmail tag got no feedback and just assumed everything went ok After some more tries the connections started to work fine when ipchains were disabled, everything worked at the first try. What makes me think that command-line sendmail works better is the fact that i could send an email using it regardless of ipchains status. It is most likely a problem external to Zope, bu having a <sendmail> tag that internally uses /usr/sbin/sendmail would be an option for cases like this. ------------------ Hannu
participants (4)
-
Hannu Krosing -
Jeff K. Hoffman -
Jim Fulton -
news-list.zope.dev＠innominate.de