Alerting

create alert when http error code from a given IP reaches a threshold

cscchen
New Member

Hi,

I'd like to create alert for the following scenario:

in http access log, we have different return codes 2xx,3xx,4xx,5xx, etc. I'd like to get an alert when within a 10 minutes windows, the returned code 4xx for an source IP is higher than 10%. The alert will include all the source IPs that meet the percentage requirement as well as the percentage of 4xx. The alert would show sth. like the following:

source ip percentage of 4xx
a.b.c.d 20%
x.x.x.x 13%

One difficulty here is that new IPs keep showing up. Would really appreciate if anyone can help.

Thanks.

Tags (2)
0 Karma

chris
Motivator

Hi,

you could use a search similar to this (the status field contains the http status code an host would contain the ip for your example):

sourcetype="access_combined" | eval status_category=substr(status,0,1)+"xx" | stats count as total count(eval(status_category="4xx")) as bad_status by host | eval Percentage=100*round(bad_status/total,2) | where Percentage>10

Schedule the search to run in realtime over a window of 10 minutes and create one alert per result.

If you need help creating the alert let me know. If you need help with the search -> post some example events and let me know if the source ip & status are extracted as fields (if you do not know what a field is yet I'll explain).

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...

Move CrowdStrike FDR ingestion to Data Inputs with a guided migration

CrowdStrike Falcon Data Replicator data is essential for security investigations, detections, and response. It ...