Predict Command - Alert when value breaches upper95

[MikeElliott]

Hi All,

I'm trying to write a search that looks at creating an alert where there is a significant spike in HTTP POST requests.

I am interested in using the predict command and alerting where the total count(http_request) (where http_request=POST) requests by source_ip breaches the predicted upper95.

In theory, it would look something like:

index=web_proxy 
| search http_request=POST 
| stats count(http_request) AS POST_Count by source_ip 
| predict POST_Count by source_ip 
| where POST_Count >= upper95

Any assistance, or pointers, would be greatly appreciated.

[MikeElliott · Accepted solution]

Apologies for the delay in responding. I was able to resolve the issue with the below logic. The index has been swapped out for a generic term, ofc 😉

index="webproxy" requestmethod=POST earliest=-65m@m latest=-5m@m 
| timechart span=1m count as POST_Requests 
| predict POST_Requests as Predicted_Requests 
| rename upper95(Predicted_Results) as Ceiling 

[MikeElliott]

This is some really cool logic that can be adapted to detect all sorts of spikes - Recently we have deployed this for spikes in DNS traffic.

[dauren_akilbeko]

The predict command must be preceded by the timechart command. The predict command requires time series data.

For more info: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Predict

[MikeElliott]

Damn - I was really hoping that this wouldn't be the case...

Okay, so we can use timechart I suppose - Any suggestions on how to get the timechart to display count(http_request) as POST_Count by source_ip, or am I asking a bit much?

[dauren_akilbeko]

The problem with predict, is that you can't use wildcard. Not very efficient way to this https://answers.splunk.com/answers/661506/predict-with-wildcard.html

[MikeElliott]

Hi dauren,

Apologies for the delay in getting back. I have posted the logic I ended up going with below - Since you were definitely instrumental in getting there, if you wanna post the logic, I'd be happy to mark as an answer 🙂

[MikeElliott]

When attempting to run the above search, I get the error message External search command 'predict' returned error code 1.