[Zope-dev] Log rotation on Windows

Mark Hammond mhammond at skippinet.com.au
Mon Jan 24 20:12:33 EST 2005


Hi all,

A number of Enfold's customers have requested a reasonable logfile rotation
scheme for Zope on Windows.  Enfold would like to work on this and
contribute it back to Zope.  The intention of this mail is to find a
consensus on the general solution we should adopt, so we can supply patches
with the greatest chance of getting into the core.

Since version 2.7, Zope has moved to a logging model similar to that used by
most Linux servers.  This means Zope can rely on external tools for all
rotation, making the Zope implementation very simple - all it need to do it
re-open the logfile upon request.  Unfortunately, this model relies on the
operating system being able to rename a file while it is in use, which is
not true for Windows.

There are a number of solutions to this problem.  In the interests of
brevity, I have briefly listed a few alternatives and why they suck, then my
preferred solution is covered in more detail.

* Move to a Zope 2.6 model, where Zope itself manages all rotation logic.
This has the obvious drawback of requiring all supporting code exist in Zope
just for Windows, and all the reasons Zope dropped it in the first place.

* Use a default log name based on the date and rotation schedule.  For
example, assuming a rotation schedule of "weekly", the logfile name would
include the date of the start of the current week.  This is similar to the
scheme used by Microsoft IIS.  Drawbacks include the need to wake at
midnight to check if the name of the logfile has changed, that the scheme
leaves lots of old files around, and that logging on Windows is
significantly different than for other operating systems.

* Do nothing.  The drawback of this is that it leaves Zope on Windows a
second-class citizen, and with administrators unable to control their logs.

* Point users at the win32 "event log" logger.  This is not pratical when
the number of events being logged is high, so is probably only suitable as a
special handler for only ERROR events, for example.

My preferred model is the following alternative:

* Have the Zope logfile handler make a "snapshot" of the log, and like Zope
on Linux, rely on external tools for all  logfile rotation semantics.

When making a "snapshot", Zope simply closes the existing logfile, renames
it to a well-known name (eg, "{logname}.snapshot", and then re-opens the
original logfile name (which should no longer exist as it was just renamed).
If the rename fails (ie, the .snapshot file already exists), Zope simply
ignores the error and continues to append to the existing log-file.

This now means that "{logname}.snapshot" can be rotated by an external tool.
For demonstration purposes, the following script should offer a simple
rotation tool for Windows:

% kill --signal=USR2 zope_pid # whatever the Windows equiv. is
% sleep 5 # wait for the file to appear
% mv -f event.log.1 event.log.2
% mv -f event.log.snapshot event.log.1

You will note that this is quite similar to a script that would work on
Linux.  The key differences are that on Windows we send the signal *before*
the process, and that we make no attempt to touch the file currently being
logged to, only the snapshot just made.  Existing tools (eg logrotate)
should be capable of rotating Zope (unfortunately logrotate doesn't seem to
have a reasonable windows port available.)

Note that for simplicity, the above assumes that the signal module will work
for us on Windows.  This is probably not true, but it should be possible to
implement a win32 specific simulation.  I've avoided this so we can focus on
the log semantics.

I welcome all comments on these schemes, or any others I have not listed.

Thanks,

Mark.



More information about the Zope-Dev mailing list