You could use grep/sed, etc, and a some shell scripts (or perl or python scripts). Instead of sed and grep (and regexes, and inconsistent results, and a lot of manual work), what perhaps a lot of folks need is unified, aggregated logging that actually looks into Zope's architecture and uses mod_perl for static stuff and CGIs served in Apache. I've thought about this a bit, and it seems to me that many sites (at least bigger ones) often use a combination of technologies (as well as multi-node server farms) to serve up their site(s). As such, if you are running stuff served from Zope, Apache, CGI, other app servers, etc, you want a unified logging platform. Add to the complication HA/load-balanced/multi-server farms, and you want to be able to aggregate logs from all boxes on all server platforms to log to a unified system. The answer seems to me to be use of a relational database, mod_perl, and the example DBI Apache logger code in O'Reilly's _Writing Apache Modules_ with the following tweaks: 1 - Add persistence to the mod_perl DB connection with Apache::DBI 2 - Tweak the tables to add the following fields: box (server node number, for HA farms) mime-type user object_id (used, along with URL to determine what object is being used by X method) 3 - mime-types determined for Apache-served static pages with RegEx on URL 4 - Zope-served pages are excluded from Apache's logging, use ZSQL methods inside Zope to log (below) 5 - ZSQL method for logging is created. Logs the relevant stats to the relational database AUTHENTICATED_USER is logged to user, and <dmtl-var id> is logged to object_id - mime-type is the object's mime-type 6 - Insert <dtml-call expr="mylogmethod()"> into standard_html_header in top level (all acquired use of standard_html_header will log) 7 - Roll your own report-system using Zope, perhaps... I've already started working on something of this sort for a long-term project I have to implement logging for my company's new HA server farm. Such an approach might also be a good thing if one is using a ZEO setup. So far, I have set up the Apache side of things, logging to MySQL, and I am working on the Zope side of things. The real work, though is not in logging, but report generation, which would be a nice Zope product. Perhaps in a few months, I might have something worth showing... Any thoughts? Sean ========================= Sean Upton Senior Programmer/Analyst Web Infrastructure SignOnSanDiego.com The San Diego Union-Tribune 619.718.5241 sean.upton@uniontrib.com ========================= -----Original Message----- From: Martin Winkler [mailto:marwin98@gmx.net] Sent: Sunday, December 10, 2000 5:26 PM To: zope@zope.org Subject: [Zope] access log analyzer (analog, webalizer etc.) Hi listies, since we all use names for urls that are not "standardized" - ending with ".html", ".gif" etc - I am wondering how to do a good logfile analyzing with standard tools like those in the subject line. I am usually running zope behind a http-accelerated squid, so I've configured squid to use "emulate_httpd_log on". But how webalizer doesn't recognize whether it's an image or html output. Did someone already write a tool that converts these logfiles like this (or similar): logfileconverter reads the logfile line by line --> checks against the ZODB whether the url outputs a "jpg", "gif" or "html" --> converts the line in the logfile accordingly (e.g. "/mydir/mymethod" to "/mydir/mymethod.html" and "mydir/mypic" to "mydir/mypic.jpg") --> then saves the modified logfile to disk or pipes it to webalizer. Thanks for your input! Martin _______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )