This is two part question that deals with isolating metric data within a multi-line event where the metric identifier strings may be broken down into 2 different sections of a data sheet (log file)
The Log
The log file format includes a date/time stamp at the top of an event generation and procedes to dump information on the application in lines
A single event can exceed 23 k lines and 700 k chars. Here's a log with an isolated single event..
(FYI, this a health.log dump from an HP NNMi management server)
$wc -cl health.log
23518 700343 health.log
Here's a mock up of the problem.
2014-06-17 10:22:13,795 INFO com.hp.ov.nms.health.log NNMi System Health Report
Hostname: somehost.com
Date: 2014-06-17 10:22:11.572
Overall Status: Normal
StatePoller
Collection Manager
Policy Count = 523
....
CustomPoller
Instance Discovery
Collection Manager
Policy Count = 23
...
Part 1) Large Event Issue
I found in a post on the forum that you can modify the inputs.conf file with a 'maxchars' value so the events don't get cut. Is this the best way to handle this, or would it be better to hack up the event? My concern here is that an event is never guaranteed to be the same size and will almost always vary.
Part 2) Multi-line Parsing Issue
As you can see in the snippet above there are preceding headers that identify the metrics I'm trying to extract. These headers are not contiguous and may, or may not have additional sub headers as identified above.
StatePoller → Collection Manager → Policy Count
CustomPoller→ Instance Discovery
→ Collection Manger → Policy Count
I'm trying to understand what the best method for parsing out these different method would be.
Please let me know if I can provide any further detail.. I can send a sample log if needed.
Thanks
Eric
... View more