All Apps and Add-ons

How Anomaly and Anomalydetection commands works in splunk

chandana204
Communicator

Hi,

I have started to learning machine learning concepts and trying to imply on Splunk tool. So, i tried to use anomaly and anomalydetection search commands but i couldn't understand how these commands are working. I have gone through documentation, there i can find how to use commands but i want to know how it's working in background. Can anyone please explain in detail.

Appreciate your time

Thanks,
Chandana

1 Solution

aoliner_splunk
Splunk Employee
Splunk Employee

Hi Chandana,

Could you please say more about what's missing in the documentation? For example, the anomalydetection command docs say that it "identifies anomalous events by computing a probability for each event and then detecting unusually small probabilities. The probability is defined as the product of the frequencies of each individual field value in the event." It then goes on to explain how those field value probabilities are computed. So, if you have two fields, A="blue" and B=7, where A has the value "blue" 25% of the time and B has a histogram of values such that the bin containing 7 is 10% of the mass of the histogram, then the probability of the event would be p = 0.25 * 0.1 = 0.025.

Does that make sense? Is it not behaving as expected?

Cheers,
- Adam

View solution in original post

jcvytla
New Member

Hi @chandana204

I'm also working on similar problem. could you please guide me through solution..

0 Karma

aoliner_splunk
Splunk Employee
Splunk Employee

Hi Chandana,

Could you please say more about what's missing in the documentation? For example, the anomalydetection command docs say that it "identifies anomalous events by computing a probability for each event and then detecting unusually small probabilities. The probability is defined as the product of the frequencies of each individual field value in the event." It then goes on to explain how those field value probabilities are computed. So, if you have two fields, A="blue" and B=7, where A has the value "blue" 25% of the time and B has a histogram of values such that the bin containing 7 is 10% of the mass of the histogram, then the probability of the event would be p = 0.25 * 0.1 = 0.025.

Does that make sense? Is it not behaving as expected?

Cheers,
- Adam

buraka
New Member

Hi Adam,
How are the histogram intervals decided, is it hard coded to 10%, can we change accordingly?
And is there a threshold value where we can control/change by probability value?

0 Karma

aoliner_splunk
Splunk Employee
Splunk Employee

Hi buraka,

There are three modes. The histogram mode is controlled by the pthresh option. For the other two modes, the docs say, "When method=zscore, performs like the anomalousvalue command. When method=iqr, performs like the outlier command." Please see the corresponding docs for those commands.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Good explanation..

Perhaps you could give us your use case @chandana204

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...