-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dylan Jay wrote:
Chris McDonough wrote:
On Sat, 2005-01-29 at 05:49, Andrew Veitch wrote:
I noticed there was a thread last month on an event service for Zope. Did that reach any conclusion?
From what I can see the present products are:
- http://www.zope.org/Members/lstaffor/Xron - Zope 2.6 only, not being developed - http://zope.org/Members/naumen/NauScheduler - shipped with one of the Zope CMS's works fine with 2.7 - http://dev.legco.biz/products/ZopeScheduler and http://dev.legco.biz/products/timerservice - works fine with 2.7 too - http://cvs.zope.org/Products/Event/ - event service without a time service, not sure of development status - http://mail.zope.org/pipermail/zope-cmf/2005-January/021747.html - Dieter's project to be based on Python's sched module - http://api.nuxeo.org/CPS3/public/ CPSCore.EventServiceTool.EventServiceTool-class.html - CPS's event service - http://cvs.sourceforge.net/viewcvs.py/*checkout*/mailmanager/ mailmanager/MailCheckThread.py?rev=1.15 - our custom scheduler for MailManager - Zope X3's event service
It has proven (at least for me) to be useful to make the distinction between "event service", "scheduler", and "clock" in the past. I tend to think of each this way:
Event Service:
Framework software that allows objects to communicate between each other indirectly by sending and receiving "events". An event service's responsibility is only to receive and send messages. Note that it is not within the scope of an event service's responsibility to run tasks at timed or scheduled intervals.
Scheduler:
Software that ensures tasks may be run at timed or scheduled intervals.
Clock:
Software that intermittently "pokes" Zope, telling it to wake up and "do something" every so often. A scheduler needs a clock. A clock can be implemented either as an external source (e.g. wget every 10 seconds) or internally (by coopting the asyncore loop).
Schedulers usually require clocks, but neither clocks nor schedulers require event services. Similarly, event services require neither a clock nor a scheduler.
It would be great to some consolidation of all of this into the core. I would also strongly favour a timer service along with the event service. I know that the argument has always been that operating systems provide time services but this is not very convenient for cross platform development and on some Windows platforms it's difficult.
If we use the definitions above, I think what you want is a "scheduler" and a "clock" to be available for Zope, but not necessarily an event service. Don't want to be pedantic about it, but when you use the term "event service" it tends to mean different things to different people.
To that end, I'd suggest using the Scheduler product at http://cvs.zope.org/Products/Scheduler. Note that this product *does* currently depend on the "Event" product at http://cvs.zope.org/Products/Event but it's only by accident (the Event product is usable separately). There is an external clock implementation that ships with the Scheduler product.
I agree with you that there should be a Zope "internal" clock. I created one not too long ago, but it's a bit invasive. Read http://www.plope.com/Members/chrism/more_scheduling and http://www.plope.com/Members/chrism/scheduling_service for an overview of the strategy I was thinking of when I did it.
Personally, I think maybe the internal clock service can also be implemented as a Product and not added to Zope proper. It would be great if someone were to maybe volunteer to take the work that I did towards it and productify it.
+1 for using http://cvs.zope.org/Products/Scheduler I think the time of a non-core scheduler needs to come to an end. Its crazy the number of implementations out there.\ I'm using the zope cvs version and I've enhanced it to make it serialised and non reentrant. I'd like to check that in.
I'd want to see the rationale and the patch, first. The product keeps the schedule itself in the ZODB, which means that the transactional semantics are going to fight with several types of "serialization".
In order to simplify installation I do think a Product version of a clock would be required. Perhaps the clock can optional be promoted from outside to ensure its thread is still alive?
I'm willing to have a crack at it as long as that effort isn't going to yet another non-core scheduler/clock.
- -10 for incluing in the core any implementation which depends on the existence of a "wild" thread inside the appserver: coping with threads and async together is a *very* tricky dance, and when it goes south, you can't do anything to debug it, or fix it; all you can do is restart the appserver. Chris' ClockServer removes the need for such a thread, by hooking ZServer's mainloop to generate the "faux" request needed to kick off async processing. A "crontab" - like schedule can be driven equally well from ClockScheduler as from a separate thread. For real scalability, you need to make the long-running async processing run in a separate process (and maybe a separate machine!): in this case, you can: - use ClockServer within the dedicated appserver process - run 'wget' or another URL-based trigger against the "trigger" URL of the dedicated appserver process - run one or more of Gary Poster's "zasync" processes, which aren't appservers at all (they don't listen for HTTP/FTP, etc.); instead, they connect to the ZODB and pull tasks from a queue stored there. Getting zasync set up correctly can be tricky, but it provides the most scalability bang of any of these solutions (e.g., using Tarek Ziyade's "zasync load balancer"). Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFC3G/l+gerLs4ltQ4RAvhaAJwIpR7edzP50xkNSxDtYaoQQMcVNwCdF0az DX+0O9D3SmTOhXMk020Guk8= =eDaJ -----END PGP SIGNATURE-----