Hi there,
I am trying to use splunk to understand the alerts that are coming out of our system. We get approx 35K alerts per week. We currently use a series of bash/grep/awk scripts to process and get stats on these alerts, but I'm trying to see if it's any easier with splunk.
I've setup a number of saved searches in splunk which correspond to "known errors/warnings" which occur in our system. These are just text patterns. There will be approx 100 of these saved searches once I've finished entering them.
I'd like to build a report/chart, that we can run daily/weekly which shows:
The number of matches for each saved search across a given period.
Possibly some way of seeing a stacked histogram of matches for each saved search. This would aid in seeing correlations between different known errors.
The number of lines which do NOT match ANY saved search. This is valuable information as it points to errors/warnings that we don't know about.
I realize that this might be a big ask!
Cheers,
Ben
... View more