Alerting

anomaly detection for multiple fields

gbenor
New Member

Hi,

I would appreciate your help in implementing the following alert with Splunk and the machine-learning toolkit.

Let's start with a simple example. Suppose I have one host in my system which sends one of two predefined messages. Then, the event should consist of two fields: [_time, message]. I can use the timechart command to generate two new numerical timeseries:

  1. count of total events.
  2. count of each predefined message.

Finally, I can use the machine learning toolkit to detect outliers and anomalies.

 

Now, I would like to describe my real situation: I have an unknown number of hosts; each host may send any kind of message. A typical event looks like: [_time, host, message].

I would like to implement an outlier alert for each possible host, possible message, and for the total number of messages per host. I prefer to have a single alert for all combinations of host and message_type. In addition, I would like to have a visualization of the timeseries of each combination.

Unfortunately, I don't have a clue how to implement this task in SPL. 

A python solution may look like the following:

  1. find unique hosts.
  2. find unique messages.
  3. For host in hosts:
    1. For msg in messages:
      1. Do anomaly detection (host, msg)
    2. Do anomaly detection (host, msg_count)

 

 

 

 

 

 

 

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...