[Zope] Log aggregator?
Paul Winkler
pw_lists at slinkp.com
Fri Sep 23 17:27:03 EDT 2005
Anybody know of any software that will aggregate logs in a variety
of formats and allow me to correlate events in one log with another?
I'm getting really REALLY tired of eyeballing and grepping a multitude
of logs on different servers, mentally translating incorrect timestamps,
and trying to work out what the heck from one log corresponds to something
in another. Anytime something goes wrong, i waste a huge amount of time
on this.
Well, duh, this should be automated, right?
I googled a bit and came up with nothing useful;
then I started hacking up a python script, which I'm sure I could
do in an hour or three, but - blah, it would be a huge timesaver if
somebody had written something like this already.
I need to handle:
* Different log formats. I'm mainly interested in the zope access log
format (aka apache format) and the zope event log format.
Must tolerate malformed lines like the #!@)%* ftp lines in the zope access
log.
* System clocks being incorrect.
On some of the servers I work with, I am not the primary admin and can't do
much about the %@W#% clock being wrong. (*cough* firewall blocking
ntp for no good reason *cough*)
So I'd like the software to let me specify how far wrong
the various logs are, and compensate.
* Servers in different time zones. Rare for me, but it does come up.
* Filtering based on arbitrary strings, or (better) regular expressions,
on a per-log basis.
* Multi-megabyte logs. Sometimes i go to check a log and sysadmin has
forgotten to rotate ever...
-PW
More information about the Zope
mailing list