Logging for ZScheduler?
I need some advise for designing a flexible system for logging the activities of scheduled events. One of the motivations for developing ZScheduler was to have a platform independent means for scheduling Zope tasks. The lask of a cron on Windows platforms was a part of the problem. ZScheduler achieves this independence by using Python's threading.Wait method for all time measurement. But ZScheduler also needs to be ISP independent. That is, you should have complete scheduling capabilities for your cohosted Zope applications without having to call up the ISP and ask for system configuration changes. ZScheduler partially achieves this level of independence, by eliminating the need for cron. But, as I installed ZScheduler under my personal account at CodeIt Computing, I realized that an ISP-dependence still exists, and it has nothing to do with cron -- it has to do with the configuration of the Scheduler's logging activity. When you have a Scheduler executing applications tasks on your website behind your back, you definitely want it to keep a log of what it is doing. For ZScheduler, I chose to use the STUPID_LOG_FILE feature of Zope. However, STUPID_LOG_FILE is an environment variable. Zope accounts at CodeIt (and probably at any other Zope ISP) cannot set environmet variables. And if they could, is anyone certain how STUPID_LOG_FILE would behave in their custom cohosting adaptation of Zope? So, I need a more flexible structure for the logging aspect of ZScheduler. While the ZEvent class is subclassable by users, I haven't made the Scheduler class instantiable -- there should be only one instance -- so it's not subclassable either, and therefore you can't override its logging method. Do you see the problem? Having little experience with Python and OO applications, I don't know how others solve this problem. Can you give me any pointers? Perhaps I've identified a typical application of RIPP? -- Thanks -- Loren
On Wed, 14 Jun 2000, Loren Stafford wrote:
I need some advise for designing a flexible system for logging the activities of scheduled events.
You could just call Zope's logging system and move it out of the scope of your product. Have a look at zLOG.py or LOGGING.txt if you havn't already. Means people can extend their logging system to their hearts content with no effect on your product.
But ZScheduler also needs to be ISP independent. That is, you should have complete scheduling capabilities for your cohosted Zope applications without having to call up the ISP and ask for system configuration changes. ZScheduler partially achieves this level of independence, by eliminating the need for cron.
This could be achieved with a seperate product. A product that ties into Zope's logging infrastructure and allows viewing the log files through the web. All it would have to do is append itself to the ZLogger.loggers tuple, and the interface is pretty trivial. And of course, it means that other products can use it without modification (Logger product, Zope itself etc.) and can be extended as required without having to submit patches to you for inclusion in ZScheduler. It would be a good idea if there was a field in the ZEvent that defined the subsystem used in the zLOG call.
So, I need a more flexible structure for the logging aspect of ZScheduler. While the ZEvent class is subclassable by users, I haven't made the Scheduler class instantiable -- there should be only one instance -- so it's not subclassable either, and therefore you can't override its logging method. Do you see the problem?
Having little experience with Python and OO applications, I don't know how others solve this problem. Can you give me any pointers?
Its not a problem with ZScheduler, it a problem that no one has written a plug-in logging system that is good enough for what you are trying to do. The existing zLOG API is fine (well - it could be better), but just needs someone to write the relevant modules. Hmm... I think I'll add a section to the Interfaces Wiki... -- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
-----Original Message----- From: zope-dev-admin@zope.org [mailto:zope-dev-admin@zope.org]On Behalf Of Stuart 'Zen' Bishop Sent: Wednesday, June 14, 2000 20:58 To: Loren Stafford
On Wed, 14 Jun 2000, Loren Stafford wrote:
I need some advise for designing a flexible system for logging the activities of scheduled events.
You could just call Zope's logging system and move it out of the scope of your product. Have a look at zLOG.py or LOGGING.txt if you havn't already. Means people can extend their logging system to their hearts content with no effect on your product.
ZScheduler already uses zLOG.LOG. I guess I forgot that it permits implementations other than stupid_log. So I guess ZScheduler is as flexible as can be, given the current API.
But ZScheduler also needs to be ISP independent. That is, you should have complete scheduling capabilities for your cohosted Zope applications without having to call up the ISP and ask for system configuration changes.
This could be achieved with a seperate product. A product that ties into Zope's logging infrastructure and allows viewing the log files through the web. All it would have to do is append itself to the ZLogger.loggers tuple, and the interface is pretty trivial. And of course, it means that other products can use it without modification (Logger product, Zope itself etc.) and can be extended as required without having to submit patches to you for inclusion in ZScheduler.
Aha! Even tho a cohosted account might not be able to modify the source of the Zope implementation directly, installing a CustomLog product could do so indirectly, by importing ZLogger and modifying the loggers tuple. To know where to write the log file, CustomLog would have to either know or be configurable enough to adapt to the ISP's Zope configuration. I suppose an absolute path property would be suffient. An ISP could preconfigure and preinstall the CustomLog product for all users, or each user could do it.
It would be a good idea if there was a field in the ZEvent that defined the subsystem used in the zLOG call.
I didn't follow your point here. By "subsytem" do you mean which logger in the loggers tuple? Then do you mean that different ZEvents could log to different loggers? Why would this be a "good idea", I mean, do you have a use case in mind? -- Thanks -- Loren
On Thu, 15 Jun 2000, Loren Stafford wrote:
It would be a good idea if there was a field in the ZEvent that defined the subsystem used in the zLOG call.
I didn't follow your point here. By "subsytem" do you mean which logger in the loggers tuple? Then do you mean that different ZEvents could log to different loggers? Why would this be a "good idea", I mean, do you have a use case in mind?
from zLOG.py: def LOG(subsystem, severity, summary, detail='', error=None, reraise=None): The first argument specifies a subsystem, which is passed to the logging implementation. A logging subsystem may choose to ignore log messages from particular subsystems, or perform special actions (eg. if a critical error has occured in the ZScheduler subsystem, page the sysadmin). By allowing an individual ZEvent to override the subsystem reported, you can gain even more control. -- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
Not knowing what subsystem was for, I just filled it with __name__. So a log entry looks like this: ------ 2000-06-14T20:05:22 INFO(0) Products.ZScheduler.Loggerr Trigger event: http://eagle:8080/zev3 Trigger time: 1970/12/31 16:00:00 US/Pacific Bang! ------ If you need to parse this, you can determine that it came from ZScheduler, you can determine which event was triggered, and you can see the output of the event, if any (Bang!, in this case). Is that good enough for any applications you have in mind? -- Loren
-----Original Message----- From: zope-dev-admin@zope.org [mailto:zope-dev-admin@zope.org]On Behalf Of Stuart 'Zen' Bishop Sent: Thursday, June 15, 2000 13:52 To: Loren Stafford Cc: Loren Stafford; zope-dev Subject: RE: [Zope-dev] Logging for ZScheduler?
On Thu, 15 Jun 2000, Loren Stafford wrote:
It would be a good idea if there was a field in the ZEvent that defined the subsystem used in the zLOG call.
I didn't follow your point here. By "subsytem" do you mean which logger in the loggers tuple? Then do you mean that different ZEvents could log to different loggers? Why would this be a "good idea", I mean, do you have a use case in mind?
from zLOG.py: def LOG(subsystem, severity, summary, detail='', error=None, reraise=None):
The first argument specifies a subsystem, which is passed to the logging implementation. A logging subsystem may choose to ignore log messages from particular subsystems, or perform special actions (eg. if a critical error has occured in the ZScheduler subsystem, page the sysadmin). By allowing an individual ZEvent to override the subsystem reported, you can gain even more control.
-- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
Loren Stafford wrote:
Not knowing what subsystem was for, I just filled it with __name__. So a log entry looks like this:
------ 2000-06-14T20:05:22 INFO(0) Products.ZScheduler.Loggerr Trigger event: http://eagle:8080/zev3 Trigger time: 1970/12/31 16:00:00 US/Pacific Bang!
------
This is a fine usage of subsystem. The idea is to identify which component logged the message. __name__ does that pretty well. In the past, we've just used a simple string, like 'zdeamon'. Does anyone object to making __name__ the convention? I like it. It lets you find the object in the code and tell you where to set breakpoints in the debugger pretty clearly without having to hunt. But perhaps there is something more interesting in some cases, like the physical location of an object in the database if it is persistent. I kinda like: subsys = string.join(self.getPhysicalPath(), '/') + ":" + `self` This is probably 2.2 specific. -- -Michel Pelletier http://www.zope.org/Members/michel/MyWiki Visit WikiCentral for the latest Zen: http://www.zope.org/Members/WikiCentral
On Thu, 15 Jun 2000, Loren Stafford wrote:
Aha! Even tho a cohosted account might not be able to modify the source of the Zope implementation directly, installing a CustomLog product could do so indirectly, by importing ZLogger and modifying the loggers tuple. To know where to write the log file, CustomLog would have to either know or be configurable enough to adapt to the ISP's Zope configuration. I suppose an absolute path property would be suffient. An ISP could preconfigure and preinstall the CustomLog product for all users, or each user could do it.
Dump it Globals.data_dir unless a given environment variable is set. (Sorry bout the double reply) -- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
I've created a STUPID_LOG helper product, called FrozenLogger, that does the minimum necessary for me to create a STUPID_LOG_FILE at my account at CodeIt Computing (where lack of telnet keeps me from setting environment variables). http://www.zope.org/Members/lstaffor/FrozenLogger The core of this is: from Globals import data_dir if os.environ.has_key('STUPID_LOG_FILE'): pass else: os.environ['STUPID_LOG_FILE']=string.join((data_dir,'FrozenLogger.txt'),'/') It works for me at CodeIt, but I doubt if it's general enough for some of the Zope virtual hosting schemes I've read about here. Let me know how your mileage varies. -- Loren ----- Original Message ----- From: "Stuart 'Zen' Bishop" <zen@cs.rmit.edu.au> To: "Loren Stafford" <lstafford@icompression.com> Cc: "Loren Stafford" <lstaffor@dynalogic.com>; "zope-dev" <zope-dev@zope.org> Sent: June 15, 2000 01:57 PM Subject: RE: [Zope-dev] Logging for ZScheduler?
On Thu, 15 Jun 2000, Loren Stafford wrote:
Aha! Even tho a cohosted account might not be able to modify the source of the Zope implementation directly, installing a CustomLog product could do so indirectly, by importing ZLogger and modifying the loggers tuple. To know where to write the log file, CustomLog would have to either know or be configurable enough to adapt to the ISP's Zope configuration. I suppose an absolute path property would be suffient. An ISP could preconfigure and preinstall the CustomLog product for all users, or each user could do it.
Dump it Globals.data_dir unless a given environment variable is set.
(Sorry bout the double reply)
-- Stuart Bishop Work: zen@cs.rmit.edu.au Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au Computer Science, RMIT University
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (4)
-
Loren Stafford -
Loren Stafford -
Michel Pelletier -
Stuart 'Zen' Bishop