If the logwatch output is itself structured as a logfile, then this becomes a simple matter of pushing this data into splunk, possibly arranging to use a sinkhole for disk management goals.
If the logwatch output is structured as a document, then you have a good deal of work to get something useful. I see http://www.cyberciti.biz/faq/freebsd-unix-log-analyzer-configuration/ suggesting that the logwatch output is actually a series of discrete components each with custom formatting, which are totally heterogeneous.
If that were my report format, and this were my problem, I would write a script that cuts up the reports, and creates a logfile for each category (logwatch_pam_auth.log), manually parsing the Date Range Processed for each report, and inserting the date between each entry.
At that point getting them as a set of events in splunk is easy.
There are nonscript approaches to getting the events cut up in splunk as well, but might require creating a custom datetime.xml and complex rules for event parsing. However, I suspect giving each data category its own sourcetype will greatly aid in sane field extraction.
Mostly though, you're using a tool that's designed to manage a single system, and isn't built for managing systems in aggregate, and struggling with that. Maybe there are better tools?
Personally I recommend Splunk for this, syslog datarates are usually low enough that it's not a real issue. Any sort of application or appliance data is usually vastly larger. However there are specialized tools which process log data to produce aggrate information. For example http://ossec.net
... View more
