Alerting

How to create an alert based on these data assurance scenarios?

indianhans
Engager

Hi All,

I am seeking some thoughts to implement data assurance. I wish to build an alerting mechanism for following circumstances :

1) If a source stops sending the logs.
2) If there is any delay in Log indexing.
** 3) If the log format at source is changed. / Log pattern is changed.

Is there any easy solution to alert on above scenarios, especially Case 3 (Log pattern change).

Regards
Rishi

0 Karma
1 Solution

woodcock
Esteemed Legend

For the first 2, you should be able to find existing searches in your MC ( Settings -> Montioring Console ). Also for the 2nd one, you can track avg(_indextime - _time) over time by index, host, and sourcetype. For the last one, you need to track values(punct) or dc(punct) or | cluster | stats count over time by index, host, and sourcetype.

The cluster command is what drives the patterns tab:
http://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Cluster

View solution in original post

woodcock
Esteemed Legend

For the first 2, you should be able to find existing searches in your MC ( Settings -> Montioring Console ). Also for the 2nd one, you can track avg(_indextime - _time) over time by index, host, and sourcetype. For the last one, you need to track values(punct) or dc(punct) or | cluster | stats count over time by index, host, and sourcetype.

The cluster command is what drives the patterns tab:
http://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Cluster

View solution in original post

indianhans
Engager

Thanks for your valuable inputs. I am already trying my hands on "Punct / Cluster". Can you please suggest some other ways, so that I can compare the results and accuracy ?

Thanks again. 🙂

0 Karma

woodcock
Esteemed Legend

Using punct is quick and dirty, using cluster is highly configurable and nuanced. Those are your options.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.