Alerting

anomaly detection for multiple fields

gbenor
New Member

Hi,

I would appreciate your help in implementing the following alert with Splunk and the machine-learning toolkit.

Let's start with a simple example. Suppose I have one host in my system which sends one of two predefined messages. Then, the event should consist of two fields: [_time, message]. I can use the timechart command to generate two new numerical timeseries:

  1. count of total events.
  2. count of each predefined message.

Finally, I can use the machine learning toolkit to detect outliers and anomalies.

 

Now, I would like to describe my real situation: I have an unknown number of hosts; each host may send any kind of message. A typical event looks like: [_time, host, message].

I would like to implement an outlier alert for each possible host, possible message, and for the total number of messages per host. I prefer to have a single alert for all combinations of host and message_type. In addition, I would like to have a visualization of the timeseries of each combination.

Unfortunately, I don't have a clue how to implement this task in SPL. 

A python solution may look like the following:

  1. find unique hosts.
  2. find unique messages.
  3. For host in hosts:
    1. For msg in messages:
      1. Do anomaly detection (host, msg)
    2. Do anomaly detection (host, msg_count)

 

 

 

 

 

 

 

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...