Alerting

Trigger on condtions met per second

kestasm
Path Finder

I am looking for possibility to be able to alert on unique source IPs within web logs, which make constant requests (GETs or POSTs) and gets server response ("403" or "407") as frequent as 20 times per 5 seconds. So for the output I want to have all the source IPs and destination matched where these conditions are met.

Tags (2)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Are you looking for "IPs that exceeded 20 requests during some five second window" or for "IPs that average over 20 requests per five seconds (= four requests per second) over an arbitrary window"?

Assuming the latter, try this:

index=proxy http_status=403 OR http_status=407 | stats count by src_ip | addinfo | eval timerange = info_max_time - info_min_time | fields - info_* | eval rate = count / timerange | where rate > 4

The former is a bit more tricky if you want statistical accuracy. For example, it'd be easy to split each minute into twelve five-second buckets and count for each bucket. However, if you have 15 events at :04 and 15 events at :06 that'd statistically be "more than 20 in five seconds" but they'd be in different buckets so that would not trigger. If that inaccuracy is fine then use something like this:

index=proxy http_status=403 OR http_status=407 | bucket span=5s _time | stats count by _time src_ip | where count > 20

That would give you potentially splunkloads of rows per src_ip, consider adding another stats at the end if you want one row per host.

If that statistical accuracy is not fine then let me know and I'll cook up something more complicated but maybe more accurate.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Are you looking for "IPs that exceeded 20 requests during some five second window" or for "IPs that average over 20 requests per five seconds (= four requests per second) over an arbitrary window"?

Assuming the latter, try this:

index=proxy http_status=403 OR http_status=407 | stats count by src_ip | addinfo | eval timerange = info_max_time - info_min_time | fields - info_* | eval rate = count / timerange | where rate > 4

The former is a bit more tricky if you want statistical accuracy. For example, it'd be easy to split each minute into twelve five-second buckets and count for each bucket. However, if you have 15 events at :04 and 15 events at :06 that'd statistically be "more than 20 in five seconds" but they'd be in different buckets so that would not trigger. If that inaccuracy is fine then use something like this:

index=proxy http_status=403 OR http_status=407 | bucket span=5s _time | stats count by _time src_ip | where count > 20

That would give you potentially splunkloads of rows per src_ip, consider adding another stats at the end if you want one row per host.

If that statistical accuracy is not fine then let me know and I'll cook up something more complicated but maybe more accurate.

0 Karma

gibba
Path Finder

Hi

you can use the access_combine source type like apache web log, you have automatically a field http_request for te get and post, e field source ip.
after you parsing your data you can structure your search like

"index=weblog http_request=403 http_request=407 | table source http_request"

after, click "save as" option alert

0 Karma

kestasm
Path Finder

Sorry I wasnt clear enough, I already have a query with conditions:
index="proxy" http_status="40*" | stats count values(host) by _time | where count > 100 | convert ctime(_time) AS time

I want to get triggered on every unique IP which exceeds (POST, GET) requests of 20 attempts per 5 seconds and only those which get "403" or "407" reply from server.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...