[Zope-dev] Running methods on Zope startup.

[grahamd@dscpl.com.au]

On Apr 25 00:57, "Chris McDonough" <chrism@zope.com> wrote:
>
> Subject: Re: [Zope-dev] Running methods on Zope startup.
>
> Something like this is slated for Zope 2.6.  If possible (and that's a big
> if at the moment, with the work we've got lined up), it is my intention to
> create a service which does the following:
> 
> ...
> 

> The two things that I think are different about this than what you specified
> is that in my conception, the startup folder is not a persistent Zope
> object, and you would need to place files in a directory on disk to hook the
> startup process.  It also solves the problem of needing to have a handle to

The important thing as far as I can see is that the hook is at least
there in such a way that the application context is available. As long
as one has that, it would then be very easy to add in a method which went
looking for the "Startup" folder in the database and went from there.

> I think your idea of registering a callback to ensure that startup is
> complete before performing any actions is very sound.  I think a callback
> can be registered almost anywhere, but a plausible place to create the
> registration method might be in lib/python/OFS/Application.py.  Then the
> callback should probably be called by lib/python/Zope/__init__.py (somewhere
> after "OFS.Application.initialize(c)", after which point all Products should
> be initialized).

Yes, worked out where it could be done, for now I was trying to work out
how it might be done without modifying the Zope code.

> This is a little brittle, though.  A better way to do this would be to
> create a generic Zope scheduling module.  This is where hooking the asyncore
> mainloop would be very appropriate.  Unfortunately, the implementation is a
> bit more complex because you'd probably need to make sure the scheduler
> didn't hold up the mainloop, and that means kicking off one or more threads
> to do the work instead of using the main thread just at startup.  However,
> it would be much more generally useful and forward-thinking to implement it
> this way, even if it was a stub that didn't actually do anything but startup
> stuff at first.

As long as one has the startup hook, a scheduler can then be a Zope product.
Steps would then be for someone to install "Startup" product and if need be
manually register callback in internal Zope startup directory to execute
"Startup" folder in known location. Someone installing the "Scheduler"
product would then register instance of it with "Startup", or it could even
be automatic in some way by using registration ability of ZCatalog.

In respect of a separate thread, I agree. I actually already have a full
scheduler/dispatcher written I can hook in. It actually has its own select
loop and thus must be run in a separate thread. It even has mechanism to
be woken up from select if new job was scheduled to be running but dispatcher
was otherwise blocking. The only issue with this separate dispatcher is
that the core is actually written in C++ and isn't pure Python but relying
on Python wrappers that already exist.

> If I were to do this, I would:
> 
>   - Implement a stub scheduler module that was really dumb and only
>     knew how to do "run once, now" jobs (no run-many jobs, no run-then
>     jobs).  Do this so as to not go insane trying to reinvent cron and at
>     at the moment. ;-)

I already have the cron code in the dispatcher system I have. :-)

>   - Implement a non-Product "Startup" module that did what I explain above,
>     but registered a callback with the scheduler instead of with
>     the method called by Zope/__init__.
> 
>   - Change medusa's asyncore so that the "poll" (and poll2, and poll3)
>     function kept track of when the scheduler had last been run.

How is that important if it is run in a separate thread?

>   - In asyncore, when either the select call times out or the last time the
>     schduler was run was over "X" seconds in the past, call in to
>     the scheduler, telling it to wake up, modifying the "last run" time.

Hmmm, still not sure why, but then the dispatcher code I have probably
has a much more developed job model than asyncore does. Not only does it
support i/o activity, timeouts, alarms, cron alarms, it also has three
different types of jobs, ie., priority, standard and idle. Add to that that
it has a separate thread, not sure why they would need to communicate.

>   - The (dumb) scheduler would wake up and notice that it had a job and
>     would kick off a thread which would call back in to the startup code.
>     Then it would remove the job from its queue and go on with its life,
>     which would be absolutely desolate until the next time Zope started up.
> 
> This architecture would facilitate innovations (like a *real* scheduling
> facility ;-) in the future.

Which is what I am interested in now. Right now though, I am more interested
in what is using the dispatcher in parallel to Zope as opposed to scheduling
persistent methods in Zope. Rather get something working first before I go
into it further. ;-)

Anyway, you have given me some ideas. Knowing that this is sort of planned
at least might at least get me motivated to hack in the Zope code a bit to
get something working and try out a few ideas.

--
Graham Dumpleton (grahamd@dscpl.com.au)