Alerting

anomaly detection for multiple fields

gbenor
New Member

Hi,

I would appreciate your help in implementing the following alert with Splunk and the machine-learning toolkit.

Let's start with a simple example. Suppose I have one host in my system which sends one of two predefined messages. Then, the event should consist of two fields: [_time, message]. I can use the timechart command to generate two new numerical timeseries:

  1. count of total events.
  2. count of each predefined message.

Finally, I can use the machine learning toolkit to detect outliers and anomalies.

 

Now, I would like to describe my real situation: I have an unknown number of hosts; each host may send any kind of message. A typical event looks like: [_time, host, message].

I would like to implement an outlier alert for each possible host, possible message, and for the total number of messages per host. I prefer to have a single alert for all combinations of host and message_type. In addition, I would like to have a visualization of the timeseries of each combination.

Unfortunately, I don't have a clue how to implement this task in SPL. 

A python solution may look like the following:

  1. find unique hosts.
  2. find unique messages.
  3. For host in hosts:
    1. For msg in messages:
      1. Do anomaly detection (host, msg)
    2. Do anomaly detection (host, msg_count)

 

 

 

 

 

 

 

 

Labels (1)
0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!