Splunk Search

Show error details when errors 10% higher than previous 30 mins?

MikeyD100
Explorer

Hi, 

I want to display the error details in the last 30 mins, so they can be investigated, when the amount of errors has increased by 10% from the previous 30mins. 

Search 1
This is the search for the data I want to show in the results 

 

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*")
| table ReqReceivedTimestamp, APIName, ReqUrl, ShopName, ResponseCode, FailureReason, FailureServiceCalloutResponse

 


Search 2
This is the search I have to work out if there are over 10% compared to the last 30 mins

 

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*")
| timechart span=30m count as server
| streamstats window=1 current=f values(server) as last30
| eval difference=server-last30
| eval percentage_change=round((difference/last30)*100,2)
| eval AboveThreshold=if(round(((server-last30)/last30),4)>.10, "True", null())
| where AboveThreshold = "True"
| table percentage_change

 


I want to understand what is the best way to get these 2 searches combined and show the table from Search 1 when  Search 2 >10%

Labels (2)
Tags (2)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

The second search is much too complicated for the task at hand.  Combine the two with an extremely literal interpretation of your requirement.

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*") earliest=-60m
| eventstats count sum(eval(if(_time < relative_time(now(), "-30m"), 1, 0))) as previous_30m
| eval show_detail = if(count > previous_30m * 2.1, "yes", null()) ``` current > 1.1 * previous ==> (current + previous) > 2.1 * previous ```
| where isnotnull(show_detail) AND _time > relative_time(now(), "-30m") ``` only show current period ```
| table ReqReceivedTimestamp, APIName, ReqUrl, ShopName, ResponseCode, FailureReason, FailureServiceCalloutResponse

 

View solution in original post

yuanliu
SplunkTrust
SplunkTrust

The second search is much too complicated for the task at hand.  Combine the two with an extremely literal interpretation of your requirement.

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*") earliest=-60m
| eventstats count sum(eval(if(_time < relative_time(now(), "-30m"), 1, 0))) as previous_30m
| eval show_detail = if(count > previous_30m * 2.1, "yes", null()) ``` current > 1.1 * previous ==> (current + previous) > 2.1 * previous ```
| where isnotnull(show_detail) AND _time > relative_time(now(), "-30m") ``` only show current period ```
| table ReqReceivedTimestamp, APIName, ReqUrl, ShopName, ResponseCode, FailureReason, FailureServiceCalloutResponse

 

MikeyD100
Explorer

Hi @yuanliu thanks, this example is much more straight forward. I have used that as guide and changed my code to meet my needs and this seems to work well now

 

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*") earliest=-60m
| eventstats count sum(eval(if(_time > relative_time(now(), "-30m"), 1, 0))) as current_30m  ```Get current```
| eventstats count sum(eval(if(_time < relative_time(now(), "-30m"), 1, 0))) as previous_30m ```Get previous```
| eval show_detail = if(round(((current_30m-previous_30m)/previous_30m),4)>0.1, "True", null()) ```Get percentage change compared to previous 30 mins```
| where isnotnull(show_detail) AND _time > relative_time(now(), "-30m")
| table ReqReceivedTimestamp, APIName, ReqUrl, ShopName, ResponseCode, FailureReason, FailureServiceCalloutResponse

 

 

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@MikeyD100 - Kindly accept the answer that helped you resolve your query by clicking on "Accept as Solution", this helps other users in the community.

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@MikeyD100 - Sorry I haven't looked at your second search closely. I'm expecting that is working as you need and give a solution to combine these searches.

index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*")

| search [index=myindex source=mysource sourcetype=mysourcetype FailureReason IN ("*Error1*", "*Error2*", "*Error3*")
| timechart span=30m count as server
| streamstats window=1 current=f values(server) as last30
| eval difference=server-last30
| eval percentage_change=round((difference/last30)*100,2)
| eval AboveThreshold=if(round(((server-last30)/last30),4)>.10, "True", null())
| eval sourcetype=if(AboveThreshold="True", "SEARCH_SOMETHING_RANDOM_RANDOM_RANDOM", "*")
| table sourcetype]

| table ReqReceivedTimestamp, APIName, ReqUrl, ShopName, ResponseCode, FailureReason, FailureServiceCalloutResponse

 

I hope this works as you expects.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...