Splunk Search

Custom Condition Search for Alert for HTTP status code report

DavidGuarneri
Path Finder

I have the following search in an alert that triggers every 15 minutes:

source="C:\logs\path\*.log" | chart count over http_status_code

http_status_code is a custom field. The search works well, and gives the nicely formatted output below via html-formatted email:

http_status_code count
200 432
400 3
401 4

I don't want to change the output of this report. It's perfect the way it is. I only want to set the condition to only send a notification if 400 gives a count of 1 or more.

I've searched for some time "splunk custom condition" for an answer on this, but the question is usually extremely complicated, and often the answer requires that the output of the report be changed. I've also searched the documentation. (So please no links with vague hints, I've seen them all).

I've tried the following, but the syntax does not appear to be correct. I think it should be a fairly simple deal, but apparently not. Help is appreciated.

http_status_code=400 | search eventcount>=1

Tags (3)
0 Karma
1 Solution

linu1988
Champion

Hi David,

You can use this

source="C:\logs\path\*.log" http_status_code=400|  where count > 0

I presume count is a field. A real-time alert or a scheduled alert will do the job.

Or you have to do the below.

source="C:\logs\path\*.log" http_status_code=400|  Stats Count(http_status_code) as Error_Count|where Error_Count > 0

View solution in original post

zeroactive
Path Finder

Define your base search just as you have:

source="C:\logs\path*.log" | chart count over http_status_code

Create an Advanced Conditional Alert to "Trigger if" field value of "Custom condition is met" and enter the follow for the condition text:

search http_status_code=401 | search count >= 1

I think that should work, but you might have to tweak it a bit.

DavidGuarneri
Path Finder

Unfortunately, the alert of all status codes did not fire after a test log entry was created 😞 . A simpler alert that only reports the count of 400 errors did fire, however. By the way, the 401 is a typo on my part, which I corrected. Should be 400.

0 Karma

DavidGuarneri
Path Finder

That looks closer to what I'm trying to do. I will try it, thank you!

0 Karma

linu1988
Champion

Hi David,

You can use this

source="C:\logs\path\*.log" http_status_code=400|  where count > 0

I presume count is a field. A real-time alert or a scheduled alert will do the job.

Or you have to do the below.

source="C:\logs\path\*.log" http_status_code=400|  Stats Count(http_status_code) as Error_Count|where Error_Count > 0

DavidGuarneri
Path Finder

I've already created the simple 400 error real time alert. That one works. As far as the quarter hour report, I've thought of some workarounds, including having the first alert trigger the second one by using a script that changes a file with a flag. This seems very roundabout, however. There has to be a way to get the quarter hour report to fire only when the required conditions exist.

0 Karma

DavidGuarneri
Path Finder

The above didn't give any results. It gives results only if I type up to "as Error_Count"; however, it only shows the count of 400 errors.

I need a custom condition because the chart gives a larger data set than the condition that should fire the alert.

The requirements are to fire one simple alert the minute a 400 error appears for a certain application. After that, a quarter hour report of ALL http status codes broken down by application should fire. I've made both reports successfully, I just need the quarter hour report to fire only when that one application gets a 400 error.

0 Karma

linu1988
Champion

Why do you need a custom condition?

111011source="C:\logs\path*.log" http_status_code=400| Stats Count(Http_status_code) as Error_Count|where Error_Count > 0|Table Http_status_code,Error_Count

will give you the alert. The above is the original search. Then you can go results > 0.

And if you really want to give the custom condition you can give count >= 1

0 Karma

DavidGuarneri
Path Finder

I tried putting that in the custom condition field. I get the following error: "Encountered the following error while trying to update: In handler 'savedsearch': Cannot parse alert condition. Unknown search command 'source'."

0 Karma

DavidGuarneri
Path Finder

Thanks for the response. I already had what you show in your first example as a separate alert. For this report, can I use this search in the custom condition search field? I don't want to change the format of the report. Thanks.

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...