Splunk Search

Predict Command - Alert when value breaches upper95

MikeElliott
Communicator

Hi All,

I'm trying to write a search that looks at creating an alert where there is a significant spike in HTTP POST requests.

I am interested in using the predict command and alerting where the total count(http_request) (where http_request=POST) requests by source_ip breaches the predicted upper95.

In theory, it would look something like:

index=web_proxy 
| search http_request=POST 
| stats count(http_request) AS POST_Count by source_ip 
| predict POST_Count by source_ip 
| where POST_Count >= upper95

Any assistance, or pointers, would be greatly appreciated.

1 Solution

MikeElliott
Communicator

Apologies for the delay in responding. I was able to resolve the issue with the below logic. The index has been swapped out for a generic term, ofc 😉

index="webproxy" requestmethod=POST earliest=-65m@m latest=-5m@m 
| timechart span=1m count as POST_Requests 
| predict POST_Requests as Predicted_Requests 
| rename upper95(Predicted_Results) as Ceiling 

View solution in original post

0 Karma

MikeElliott
Communicator

Apologies for the delay in responding. I was able to resolve the issue with the below logic. The index has been swapped out for a generic term, ofc 😉

index="webproxy" requestmethod=POST earliest=-65m@m latest=-5m@m 
| timechart span=1m count as POST_Requests 
| predict POST_Requests as Predicted_Requests 
| rename upper95(Predicted_Results) as Ceiling 
0 Karma

MikeElliott
Communicator

This is some really cool logic that can be adapted to detect all sorts of spikes - Recently we have deployed this for spikes in DNS traffic.

0 Karma

dauren_akilbeko
Communicator

The predict command must be preceded by the timechart command. The predict command requires time series data.

For more info: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Predict

MikeElliott
Communicator

Damn - I was really hoping that this wouldn't be the case...

Okay, so we can use timechart I suppose - Any suggestions on how to get the timechart to display count(http_request) as POST_Count by source_ip, or am I asking a bit much?

0 Karma

dauren_akilbeko
Communicator

The problem with predict, is that you can't use wildcard. Not very efficient way to this https://answers.splunk.com/answers/661506/predict-with-wildcard.html

0 Karma

MikeElliott
Communicator

Hi dauren,

Apologies for the delay in getting back. I have posted the logic I ended up going with below - Since you were definitely instrumental in getting there, if you wanna post the logic, I'd be happy to mark as an answer 🙂

0 Karma

MikeElliott
Communicator

When attempting to run the above search, I get the error message External search command 'predict' returned error code 1.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...