Use Mysql results to determine log deviation

[keithtyler]

**My mission: Alert networking staff when one of their devices has high log deviation.

**How I think it should be done: Using the tacacs database since it is always up to date.
I am selecting a list of IP's from our Tacacs database.
mysqlquery spec=mysql.companyxyz.net query="select ip FROM host WHERE ip > 0"

Here is the query to find log deviation for all hosts:
index=_internal group="per_host_thruput" |eval mb=kb/1024 |stats mean(eps), stdev(eps) AS "Standard Deviation", var(eps) AS "Variance EPS", sum(mb) AS "MB of Syslog Data" by series |sort 25 -"Standard Deviation"

My Question: how do I tell splunk to alert using the IP's returned from the query?

[richcollier]

The Prelert Anomaly Detective app automatically baselines event rates of any kind of event (or even the occurrences of certain field values - for example events where "field=value") and automatically points out when the event rates deviated from the observed "normal". It might be worth checking out.

[sdaniels]

You could modify your search to include a where clause for the amount of deviation. Add "| where "Standard Deviation" > value". Create an alert based on that, then Splunk will alert for values above that during a particular time window. You can have the alert pass the result set when you get an email alert.

[mikelanghorst]

If you have the host extracted as a field, it should be included in the results by default. If host isn't extracted you could either create a props entry or use |rex to extract it on the fly.

[sdaniels]

So you could do > 0...therefore any deviation from servers in a given time range will send an alert and you could have the email contain the results which would be the list of hosts.

[keithtyler]

Not concerned with amount of deviation only about hosts causing the deviation.