Dylan Jay wrote:

Chris McDonough wrote:

...

On Sat, 2005-01-29 at 05:49, Andrew Veitch wrote:

...

I noticed there was a thread last month on an event service for Zope. Did that reach any conclusion?

From what I can see the present products are:

- http://www.zope.org/Members/lstaffor/Xron - Zope 2.6 only, not being developed - http://zope.org/Members/naumen/NauScheduler - shipped with one of the Zope CMS's works fine with 2.7 - http://dev.legco.biz/products/ZopeScheduler and http://dev.legco.biz/products/timerservice - works fine with 2.7 too - http://cvs.zope.org/Products/Event/ - event service without a time service, not sure of development status - http://mail.zope.org/pipermail/zope-cmf/2005-January/021747.html - Dieter's project to be based on Python's sched module - http://api.nuxeo.org/CPS3/public/ CPSCore.EventServiceTool.EventServiceTool-class.html - CPS's event service - http://cvs.sourceforge.net/viewcvs.py/*checkout*/mailmanager/ mailmanager/MailCheckThread.py?rev=1.15 - our custom scheduler for MailManager - Zope X3's event service

It has proven (at least for me) to be useful to make the distinction between "event service", "scheduler", and "clock" in the past. I tend to think of each this way:

Event Service:

Framework software that allows objects to communicate between each other indirectly by sending and receiving "events". An event service's responsibility is only to receive and send messages. Note that it is not within the scope of an event service's responsibility to run tasks at timed or scheduled intervals.

Scheduler:

Software that ensures tasks may be run at timed or scheduled intervals.

Clock:

Software that intermittently "pokes" Zope, telling it to wake up and "do something" every so often. A scheduler needs a clock. A clock can be implemented either as an external source (e.g. wget every 10 seconds) or internally (by coopting the asyncore loop).

Schedulers usually require clocks, but neither clocks nor schedulers require event services. Similarly, event services require neither a clock nor a scheduler.

...

It would be great to some consolidation of all of this into the core. I would also strongly favour a timer service along with the event service. I know that the argument has always been that operating systems provide time services but this is not very convenient for cross platform development and on some Windows platforms it's difficult.

If we use the definitions above, I think what you want is a "scheduler" and a "clock" to be available for Zope, but not necessarily an event service. Don't want to be pedantic about it, but when you use the term "event service" it tends to mean different things to different people.

To that end, I'd suggest using the Scheduler product at http://cvs.zope.org/Products/Scheduler. Note that this product *does* currently depend on the "Event" product at http://cvs.zope.org/Products/Event but it's only by accident (the Event product is usable separately). There is an external clock implementation that ships with the Scheduler product.

I agree with you that there should be a Zope "internal" clock. I created one not too long ago, but it's a bit invasive. Read http://www.plope.com/Members/chrism/more_scheduling and http://www.plope.com/Members/chrism/scheduling_service for an overview of the strategy I was thinking of when I did it.

Personally, I think maybe the internal clock service can also be implemented as a Product and not added to Zope proper. It would be great if someone were to maybe volunteer to take the work that I did towards it and productify it.

+1 for using http://cvs.zope.org/Products/Scheduler I think the time of a non-core scheduler needs to come to an end. Its crazy the number of implementations out there.\ I'm using the zope cvs version and I've enhanced it to make it serialised and non reentrant. I'd like to check that in.

In order to simplify installation I do think a Product version of a clock would be required. Perhaps the clock can optional be promoted from outside to ensure its thread is still alive?

I'm willing to have a crack at it as long as that effort isn't going to yet another non-core scheduler/clock.

Another not listed is a Zope3 scheduler that came from a sprint thats also back ported to Z2 http://www.last-bastion.net/ZScheduler Is this what we should be using? We really really need a core scheduler or at least one product should win out somehow.

Tres Seaver wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Dylan Jay wrote:

...

+1 for using http://cvs.zope.org/Products/Scheduler I think the time of a non-core scheduler needs to come to an end. Its crazy the number of implementations out there.\ I'm using the zope cvs version and I've enhanced it to make it serialised and non reentrant. I'd like to check that in.

I'd want to see the rationale and the patch, first. The product keeps the schedule itself in the ZODB, which means that the transactional semantics are going to fight with several types of "serialization".

The cvs.zope scheduler will run more than one task at a time which isn't always desireable if they cause conflicts. I added a lock so that when notify is called, it will not run if a task is still running. Of couse this will only work on a single machine but I figure thats a fair enough comprimise. I'm pretty sure the scheduler was also rerunning the same event again if it had not finished before the next notify occured anyway so that any longrunning process was causes a conflict error. I couldn't see that these problems would cause any transactional problems. I've included the changes if you want to have a look. So is the cvs.zope scheduler the 'official' one? Will it be included in any zope releases?

...

In order to simplify installation I do think a Product version of a clock would be required. Perhaps the clock can optional be promoted from outside to ensure its thread is still alive?

I'm willing to have a crack at it as long as that effort isn't going to yet another non-core scheduler/clock.

- -10 for incluing in the core any implementation which depends on the existence of a "wild" thread inside the appserver: coping with threads and async together is a *very* tricky dance, and when it goes south, you can't do anything to debug it, or fix it; all you can do is restart the appserver.

Chris' ClockServer removes the need for such a thread, by hooking ZServer's mainloop to generate the "faux" request needed to kick off async processing. A "crontab" - like schedule can be driven equally well from ClockScheduler as from a separate thread.

I was suggesting making ClockServer easy to install. At the moment its a pain in the arse. Productise it as chris suggested. Is that not trivial? Would ClockServer be acceptable to in the core? Would the cvs scheduler be able to be changed such that ClockServer could be selected as a clock source?

For real scalability, you need to make the long-running async processing run in a separate process (and maybe a separate machine!): in this case, you can:

Not sure what scalability has to do with it. Do you mean stability? Dylan. ############################################################################## # # Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved. # # This software is subject to the provisions of the Zope Public License, # Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE # ############################################################################## """ Scheduler module. $Id: Scheduler.py,v 1.24 2003/07/10 01:42:57 tseaver Exp $ """ import os, time, sys, random from types import FloatType, IntType, StringType import threading import Globals from BTrees import IOBTree, OOBTree from Persistence import Persistent from ExtensionClass import Base from Acquisition import aq_base from AccessControl import ClassSecurityInfo from ZODB.POSException import ConflictError from OFS.SimpleItem import SimpleItem from OFS.PropertyManager import PropertyManager from Products.PageTemplates.PageTemplateFile import PageTemplateFile from Products.Event.ISubscriptionAware import ISubscriptionAware from Products.Event.ISubscriber import ISubscriber from Products.Scheduler import IScheduler from Products.Scheduler import IScheduledEvent from Products.Scheduler import IDescheduledEvent from Products.Scheduler import ITimeEvent from Products.Scheduler.Task import Task from Products.Scheduler.Task import InconsistentSchedulerMarkerTask from Products.Scheduler.SchedulerPermissions import * from zLOG import LOG, PROBLEM, ERROR, BLATHER, INFO _NotifyLock = threading.Lock() __version__ = "$Revision: 1.24 $"[11:-2] class Scheduler(SimpleItem, PropertyManager): meta_type = 'Scheduler' manage_options = ( {'label': 'Current Tasks', 'action': 'manage_current_tasks'}, {'label': 'Schedule A Task', 'action': 'manage_schedule_tasks'}, {'label': 'Properties', 'action':'manage_propertiesForm'}, {'label': 'Security', 'action':'manage_access'}, {'label': 'Ownership', 'action':'manage_owner'}, {'label': 'Undo', 'action':'manage_UndoForm'}, ) _properties = ( {'id':'steps', 'type':'lines', 'mode':'w'}, {'id':'event_service', 'type':'string', 'mode':'w'}, {'id':'filter_data', 'type':'string', 'mode':'r'}, {'id':'interval', 'type':'int', 'mode':'w'}, ) # offering to the security gods security = ClassSecurityInfo() ok = {'meta_type':1, 'id':1, 'icon':1, 'bobobase_modification_time':1 , 'title_or_id': 1, 'filter_data': 1, 'event_service': 1} security.setDefaultAccess(ok) security.setPermissionDefault(MGMT_SCREEN_PERM, ['Manager']) security.setPermissionDefault(CHANGE_SCHEDULE_PERM, ['Manager']) security.setPermissionDefault(VIEW_SCHEDULE_PERM, ['Manager']) security.setPermissionDefault(NOTIFY_SCHEDULE_PERM, ['Manager']) __implements__ = (IScheduler, ISubscriptionAware, ISubscriber) def __init__(self, id, title='', filter_data='', event_service='portal_events'): self.id = id self.title = title self.subscribed = [] self.tasks = OOBTree.OOBTree() self.times = IOBTree.IOBTree() if filter_data: self.filter_data = filter_data else: self.filter_data = '' self.event_service = event_service self.steps = [] self.interval = 0 def subscribedTo(self, subscribable, event_type, filter): """ alerts the object that it has subscribed, via a call from itself or from another object, to the subscribable. The event_type and filter match the arguments provided to ISubscribable.subscribe. """ subscribable = aq_base(subscribable) self.subscribed.append((subscribable, event_type, filter)) self._p_changed = 1 def unsubscribedFrom(self, subscribable, event_type, filter): """alerts the object that it has unsubscribed, via a call from itself or from another object, to the subscribable. The event_type and filter match the exact event_type and filter of the deleted subscription, rather than, necessarily, the arguments provided to ISubscribable.unsubscribe. """ subscribable = aq_base(subscribable) self.subscribed.remove((subscribable, event_type, filter)) self._p_changed = 1 def getFilter(self): if self.filter_data: return Filter(self.filter_data) return None def manage_afterAdd(self, item, container): event_service = getattr(self, self.event_service, None) if event_service is not None: # receive all time events event_service.subscribe(self, ITimeEvent) # receive schedule events that match our id event_service.subscribe(self, IScheduledEvent, filter=self.getFilter()) def manage_beforeDelete(self, item, container): event_service = getattr(self, self.event_service, None) if event_service is not None: # un-receive all time events event_service.unsubscribe(self, ITimeEvent) # un-receive schedule events that match our id event_service.unsubscribe(self, IScheduledEvent, filter=self.getFilter()) security.declareProtected(NOTIFY_SCHEDULE_PERM, 'notify') def notify(self, event=None, max_tasks=None): """If it is specified, 'event' must be a "time event" or a "schedule event". A time event may be signified by any of the following objects: - a float or an int (although the ability for the event to be a float or int is not specified in the IScheduler interface) - None, which is translated into a time event represented by the current time. - an object that implements the ITimeEvent interface When we receive a time event, we process all events in the queue that have a scheduled time before the time event. A schedule event is an object that implements the IScheduledEvent interface. When we receive a schedule event, we schedule the event but we do not actually process the event queue. 'max_tasks' limits the number of tasks to be dispatched; it must be a non-negative integer, or None (a string representation of the integer or "None" will be converted). If the value is None, dispatch all pending tasks; otherwise dispatch up to 'max_tasks'. """ LOG('Scheduler (%s)' % self.getId(), BLATHER, 'notify called') if event is None: # If event is None, assume that we want to run tasks before # "now" by turning event into a time.time as of now. # This is the common case. t = int(time.time()) elif (isinstance(event, IntType) or isinstance(event, FloatType)): # Check if event is an integer or a float. t = int(event) elif ITimeEvent.isImplementedBy(event): # Check if event is an object that implements the ITimeEvent # interface that we need to get a number time from t = int(event.getTime()) elif IScheduledEvent.isImplementedBy(event): # this is a schedule event, we want to schedule this event # with ourselves, but we don't want to actually perform # any tasks. self.schedule(event.getTime(), event) return elif IDescheduledEvent.isImplementedBy(event): # this is a deschedule event, we want to deschedule this event, # but we don't want to actually perform any tasks. self.deschedule(event.getTaskId()) return else: # We don't know what to do, so we punt. raise ValueError, ( '"event" must be an object that supports the ITimeEvent ' 'interface, an object that supports the IScheduledEvent ' 'interface, an integer, or a float. %s is not a valid ' 'value.' % event) if not _NotifyLock.acquire(0): LOG('Scheduler (%s)' % self.getId(), INFO, 'notify ended: reentrant') return if max_tasks is None or max_tasks == 'None': max_tasks = sys.maxint count, max_tasks = 0, int(max_tasks) for this_time, this_task, taskid in self.getPendingTasks(t): ok = 1 if count >= max_tasks: LOG('Scheduler (%s)' % self.getId(), INFO, 'Processed %d tasks; done.' % count ) break try: this_task = this_task.__of__(self) status = this_task() # perform the task except ConflictError: # allow conflict errors to permeate self.deschedule(taskid) # deschedule the task _NotifyLock.release() raise except: msg = ('The task %s call failed hard!' % this_task.getDescription()) LOG('Scheduler (%s)' % self.getId(), ERROR, msg, error=sys.exc_info()) ok = 0 count += 1 if ok and not status: # task may want to reschedule try: status = this_task.next() # does it want to reschedule? msg = 'next returned %s' % str(status) LOG('Scheduler (%s)' % self.getId() , BLATHER, msg) if not status: # if not, that's ok. ok = 0 except ConflictError: # allow conflict errors to permeate self.deschedule(taskid) # deschedule the task _NotifyLock.release() raise except: msg = ('The task %s next failed hard!' % this_task.getDescription()) LOG('Scheduler (%s)' % self.getId(), ERROR, msg, error=sys.exc_info()) ok = 0 try: next_time, next_task = status # make sure we don't try to store an acquisition wrapper next_task = aq_base(next_task) except TypeError: next_time, next_task = None, None if IScheduledEvent.isImplementedBy(next_task): self.schedule(next_time, next_task) else: msg = ('The task %s declined to reschedule itself ' 'by returning %s from its __call__' % (this_task.getDescription(), str(status))) LOG('Scheduler (%s)' % self.getId(), BLATHER, msg) #Deschedule after so others can tell its not done yet self.deschedule(taskid) # deschedule the task _NotifyLock.release() security.declareProtected(NOTIFY_SCHEDULE_PERM, 'notifyOne') def notifyOne(self): """ Dispatch a single task. Primarily a helper function for XMLRPC-driven clock. """ self.notify(max_tasks=1) security.declareProtected(NOTIFY_SCHEDULE_PERM, 'notifyMax') def notifyMax(self, max_tasks=1): """ Dispatch up to 'max_tasks' tasks. Primarily a helper function for XMLRPC-driven clock. """ self.notify(max_tasks=max_tasks) security.declareProtected(VIEW_SCHEDULE_PERM, 'getPendingTasks') def getPendingTasks(self, when=None): if when is None: when = time.time() when = int(when) # XXX: We should really be storing paths and dereferencing them # here; re-wrapping the task object in the schedule is icky! timelist = self.times.items(None, when) # min, max l = [] for t, taskid_list in timelist: for taskid in taskid_list: task = self.tasks.get(taskid) if task is None: # there is a consistency problem between the times and # tasks btrees. # this has happened in production on the bonzai # project. we don't know what has caused it yet, so # we work around it but don't fail. task=InconsistentSchedulerMarkerTask(t,taskid).__of__(self) else: task = aq_base(task).__of__(self) l.append((t, task, taskid)) return l security.declareProtected(VIEW_SCHEDULE_PERM, 'getPendingTaskInfo') def getPendingTaskInfo(self, when=None): """ Return a sequence of mappings for use by UI. """ l = [] for key, task, taskid in self.getPendingTasks(when): d = { 'when':task.getTime(), 'info':task.getInfo(), 'description':task.getDescription(), 'taskid':taskid } l.append((key, d)) return l def new_task_id(self, time): return '%010i%010i' % (time, random.randint(0, sys.maxint - 1)) security.declareProtected(CHANGE_SCHEDULE_PERM, 'schedule') def schedule(self, time, task): task = aq_base(task) taskid = self.new_task_id(time) while not self.tasks.insert(taskid, task): taskid = self.new_task_id(time) task.taskid = taskid time = int(time) l = self.times.get(time, []) l.append(taskid) self.times[time] = l return taskid security.declareProtected(CHANGE_SCHEDULE_PERM, 'deschedule') def deschedule(self, taskid): """ deschedule the task by removing from the tasks and times BTrees """ time = int(taskid[:10]) try: del self.tasks[taskid] except KeyError: # an inconsistency occurred between the tasks and times # btrees or the taskid is bogus LOG('Scheduler (%s)' % self.getId(), ERROR, 'a task with taskid %s could not be removed from the tasks ' 'btree' % taskid) l = self.times.get(time) try: l.remove(taskid) except (ValueError, IndexError, KeyError, AttributeError): # an inconsistency occurred between the tasks and times # btrees or the times entry is bogus LOG('Scheduler (%s)' % self.getId(), ERROR, 'taskid %s could not be removed from the time list for time ' '%s' % (taskid, time)) if l: # a nonempty list self.times[time] = l elif l is not None: # an empty list del self.times[time] security.declareProtected(CHANGE_SCHEDULE_PERM, 'checkConsistency') def checkConsistency(self): """ """ l = [] # check task id consistency timelist = self.times.items(None, sys.maxint - 1) for t, taskid_list in timelist: for taskid in taskid_list: task = self.tasks.get(taskid) if hasattr(task, 'taskid'): if task.taskid != taskid: l.append( ('Task registered under taskid "%s" has taskid ' 'attribute "%s"' % (taskid, task.taskid) )) else: l.append('Task registered under taskid "%s" has no taskid') # check BTree consistency from cStringIO import StringIO io = StringIO() import traceback from BTrees.check import check for tree in (self.tasks, self.times): try: check(aq_base(tree)) except: traceback.print_exc(io) io.seek(0) s = io.read() if s: l.append(s) if l: return '\n'.join(l) return 'OK' security.declareProtected(CHANGE_SCHEDULE_PERM, 'fixupTimesBTrees') def fixupTimesBtree(self): """ Make times btree consistent (recover from desync) """ newtree = IOBTree.IOBTree() timelist = self.times.items(None, sys.maxint - 1) for t, taskid_list in timelist: for taskid in taskid_list: task = self.tasks.get(taskid) if task is None: continue else: l = newtree.get(t, []) l.append(taskid) newtree[t] = l self.oldtimes = self.times self.times = newtree return 'OK' security.declareProtected(CHANGE_SCHEDULE_PERM, 'manage_scheduleTask') def manage_scheduleTask(self, description, when, path, interval, max_age, max_retries, retry_backoff_time, REQUEST=None): """ """ task = Task(description, when, path, interval, max_age, max_retries, retry_backoff_time) self.schedule(when, task) if REQUEST is not None: return self.manage_current_tasks(self, REQUEST) security.declareProtected(NOTIFY_SCHEDULE_PERM, 'manage_notifyTasks') def manage_notifyTasks(self, REQUEST=None): """ Run any pending tasks (allow poking via the ZMI).""" LOG('Scheduler (%s)' % self.getId(), BLATHER, 'manage_notifyTasks was called via manual ZMI poke.') self.notify() if REQUEST is not None: return self.manage_current_tasks(self, REQUEST) security.declareProtected(CHANGE_SCHEDULE_PERM, 'manage_descheduleTask') def manage_descheduleTask(self, taskids, REQUEST=None): """ """ if isinstance(taskids, StringType): taskids = [taskids] for t in taskids: try: # this try-except is in case the form is submitted after # notify has kicked a task out of the queue. self.deschedule(t) except KeyError: pass if REQUEST is not None: return self.manage_current_tasks(self, REQUEST) security.declarePublic('format_time') def format_time(self, t): return time.ctime(t) security.declarePublic('now') def now(self): return int(time.time()) security.declarePublic('maxTime') def maxTime(self): return sys.maxint security.declareProtected(MGMT_SCREEN_PERM, 'manage_current_tasks') manage_current_tasks = PageTemplateFile( 'www/manage_current_tasks', globals(),__name__='manage_current_tasks' ) security.declareProtected(MGMT_SCREEN_PERM, 'manage_schedule_tasks') manage_schedule_tasks = PageTemplateFile( 'www/manage_schedule_tasks',globals(),__name__='manage_schedule_tasks' ) class Filter(Base, Persistent): def __init__(self, key): self.key = key def __call__(self, event): key = getattr(event, 'getFilterData', None) if key and key() == self.key: return 1 return 0 manage_addSchedulerForm = PageTemplateFile( 'www/addScheduler', globals(), __name__='manage_addSchedulerForm' ) def manage_addScheduler(self, id, title, filter_data=None, event_service='portal_events', REQUEST=None): """ """ ob = Scheduler(id, title, filter_data, event_service) event_service = getattr(self, ob.event_service) self._setObject(id, ob) if REQUEST is not None: return self.manage_main(self, REQUEST, update_menu=1) Globals.InitializeClass(Scheduler) Globals.InitializeClass(Task)

Dylan Jay <gmane@dylanjay.com> wrote:

Tres Seaver wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

...

Chris' ClockServer removes the need for such a thread, by hooking ZServer's mainloop to generate the "faux" request needed to kick off async processing. A "crontab" - like schedule can be driven equally well from ClockScheduler as from a separate thread.

So what's wrong with including ClockServer in the core or making it easier to install? (ie not having to put packages in the python path which is hard with some hosting arrangements)

And what's the argument against a core scheduler regardless of a clock? Isn't running background tasks a common need amoungst many very different tools and therefore a interstructure issue?

Myself I'm for having ClockServer in the core, if Chris and others agree. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com

Chris McDonough wrote:

On Fri, 2005-07-22 at 13:11 +0200, Florent Guillaume wrote:

...

Dylan Jay <gmane@dylanjay.com> wrote:

...

Tres Seaver wrote:

...

Myself I'm for having ClockServer in the core, if Chris and others agree.

It's fine with me. We maybe just need to remove the C extension in it... someone (you?) provided a pure python implementation of what it does that probably runs just as fast.

There is also TimerService by Nikolay Kim. Does that work in the same way? One nice feature of this is that products such as a scheduler subscribe to its tick so that many products can benifit from the clock. This seems more flexible to me, rather than putting the path to be called in the zope.conf. Another nice feature is a plugin archetecture shown by http://www.last-bastion.net/portal_zpydoc/Products.ZScheduler.html Here one kind of clock can be replaced by another so for instance a cron job could be used instead of the clockservice. That product also had a singlethreaded clock that ensured the next tick didn't occur until the last finished. Personally I think that is the schedulers job rather than the clocks but I do think allowing an cron override is a good idea. So perhaps the archetecture could look something like ClockService (outside zope) v Clock ControlPanel (inside zope... could also be called via cron wget) v v Scheduler A Product X In other words products subscribe to the clock control panel in order to get a regular call back and the clock control panel is driver from clockserver or some other outside source. Or alternatively you could just make the clock control panel a scheduler and say that anyone wanting a clock tick has to register a recurring task.

I wouldn't be apt include the Scheduler product in the core. I think it may be a tad too complicated.

Do you mean the zope cvs scheduler? I think this is the simplest implementation I've seen. It is API friendly in that it allows for code to add events reasonably easily. Its UI could be improved somewhat. The Zscheduler above as a nicer UI that could be borrowed. I think the cvs scheduler needs to at least have an option of not allowing more than one task at a time to run. This could be implemented in the callers code but it is a nice service. At the very least the current code needs to be fixed as it causes conflicts if its notify is called before the last one has finished.

The Event product is likely superseded by the Zope 3 event system included in Five.

Is there really a need for an event system? I can't see where the actual event system is even used in the cvs scheduler. Surely a listener/talker pattern is easy enough to implement?

So to summarise: - ClockServer is a good idea to put in the core but needs to made python only. - Schedular from cvs is probably the best one to use but not neccerily in the core. Does the schedular work with ClockServer by puting the notify path in zope.conf or will the schedular need to be modified to subscribe to the clock? So whats next? Does one put a feature request in the collector these days or is there another process? Chris McDonough wrote:

On Sun, 2005-07-24 at 11:51 +1000, Dylan Jay wrote:

...

Chris McDonough wrote:

...

On Fri, 2005-07-22 at 13:11 +0200, Florent Guillaume wrote:

...

Dylan Jay <gmane@dylanjay.com> wrote:

...

Tres Seaver wrote:

...

...

Myself I'm for having ClockServer in the core, if Chris and others agree.

It's fine with me. We maybe just need to remove the C extension in it... someone (you?) provided a pure python implementation of what it does that probably runs just as fast.

There is also TimerService by Nikolay Kim. Does that work in the same way?

I don't know. As far as I know, ClockServer is the only product that works as a medusa server as opposed to a separate thread or process (which is really its only attraction).

...

One nice feature of this is that products such as a scheduler subscribe to its tick so that many products can benifit from the clock. This seems more flexible to me, rather than putting the path to be called in the zope.conf.

This is typically the domain of an event service (components subscribe to events, and the service contacts all of them when one of those events happens). I agree this is very useful but I'd try to implement it in terms of a more general event service. The clock is really just a clock and it should do not much more.

...

Another nice feature is a plugin archetecture shown by http://www.last-bastion.net/portal_zpydoc/Products.ZScheduler.html Here one kind of clock can be replaced by another so for instance a cron job could be used instead of the clockservice. That product also had a singlethreaded clock that ensured the next tick didn't occur until the last finished. Personally I think that is the schedulers job rather than the clocks but I do think allowing an cron override is a good idea. So perhaps the archetecture could look something like

ClockService (outside zope) v Clock ControlPanel (inside zope... could also be called via cron wget) v v Scheduler A Product X

In other words products subscribe to the clock control panel in order to get a regular call back and the clock control panel is driver from clockserver or some other outside source.

That'd be fine but IMHO a scheduler should probably still be implemented "on top" of one or more dumb clocks and clock ticks should cause events to fire (if necessary) based on an event system that can be used for other things.

...

Or alternatively you could just make the clock control panel a scheduler and say that anyone wanting a clock tick has to register a recurring task.

...

I wouldn't be apt include the Scheduler product in the core. I think it may be a tad too complicated.

Do you mean the zope cvs scheduler?

Yes.

...

I think this is the simplest implementation I've seen. It is API friendly in that it allows for code to add events reasonably easily. Its UI could be improved somewhat. The Zscheduler above as a nicer UI that could be borrowed. I think the cvs scheduler needs to at least have an option of not allowing more than one task at a time to run. This could be implemented in the callers code but it is a nice service. At the very least the current code needs to be fixed as it causes conflicts if its notify is called before the last one has finished.

It also currently depends on the Event product (although that would be easy to remove).

But FWIW, I'm not interested in putting any scheduler in the core personally right away, just a clock for now. This is mostly for maintenance reasons. If we did put one in, I'd probably try to implement it in Zope 3 and bridge it to Zope 2 using the Zope 3 event service and Five.

...

...

The Event product is likely superseded by the Zope 3 event system included in Five.

Is there really a need for an event system? I can't see where the actual event system is even used in the cvs scheduler. Surely a listener/talker pattern is easy enough to implement?

The Scheduler product indeed subscribes to an event service in order to accept "ITimeEvent" and "IScheduledEvent" event notifications. See its "notify" method in Scheduler.Scheduler.

The pattern is easy to implement but it is useful to have one component handle event registration and notification so you don't have to reimplement it over and over.

- C

_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dylan Jay wrote:

Tres Seaver wrote:

...

Chris' ClockServer removes the need for such a thread, by hooking ZServer's mainloop to generate the "faux" request needed to kick off async processing. A "crontab" - like schedule can be driven equally well from ClockScheduler as from a separate thread.

So what's wrong with including ClockServer in the core or making it easier to install? (ie not having to put packages in the python path which is hard with some hosting arrangements)

Hmm, I thought I was arguing *for* including ClockServer in the core; what I don't want is some of the alternatives which rely on running special threads.

And what's the argument against a core scheduler regardless of a clock? Isn't running background tasks a common need amoungst many very different tools and therefore a interstructure issue?

I could see including the Scheduler product as well, although there are some issues for "long-running" tasks which might not fit well (conflicts on the schedule itself are possible). Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFC4Qr++gerLs4ltQ4RAg/BAKC9jzLSZ/CnMpDpKm7F+p2Wbm9CTACdEXgd c4gpIBSEXNa1gu3XDfF+JbQ= =DQrV -----END PGP SIGNATURE-----