Hi   I am trying to create an alert when there are more than 10 login attempt failures by same user in last 15 mins. I have scheduled my cron to run for every 15 mins However, I have a scenario where there were 8 login failures at 8:59 and another 5 at 9:01 for same user if we run the cron at 9 and 9:15, neither of them would alert. Is it possible to manipulate some of splunk's alert options to get the behavior I want? ... View more

I was given below excel of rules FT OOS ErrorCode Priority Min_req Max_Req Min_Req_Fail Max_Req_Fail Failure% watch ALCATEL_FT ALCATEL ALLERRORS 1 20 300 20 40 0 3 ALCATEL_FT ALCATEL ALLERRORS 2 20 300 41 75 0 2 ALCATEL_FT ALCATEL ALLERRORS 3 20 300 76 0 0 1 ALCATEL_FT ALCATEL ALLERRORS 4 20 0 301 0 0 1 ALCATEL_FT ALCATEL ALLERRORS 5 20 0 0 0 100 2 GWR_FT GWRNG_NY ALLERRORS 1 20 300 20 60 0 3 GWR_FT GWRNG_NY ALLERRORS 2 20 300 61 100 0 2 GWR_FT GWRNG_NY ALLERRORS 3 20 300 101 0 0 1 GWR_FT GWRNG_NY ALLERRORS 4 20 0 201 0 0 1 GWR_FT GWRNG_NY ALLERRORS 5 20 0 0 0 90 2 I have the log file in splunk saved into index dte4fios and I have the fields FT,Error_Code & OSS which are same as above lookup file columns FT,ErrorCode & OSS. I was given below requirement to create a very dynamic alert by using the rules in the above lookup file. 1. My Alert should run every 15 mins (lete us assume alert is running at 11AM)and check for if there is any any FT & Error Code combination which has satisfied above rules and if yes then send an alert with output of some other query. 2. Let me explain the understanding of rules in above lookup file for one of the FT ALCATEL_FT in this case. Rules should be validated in order of Prioirty 5 to Proirity 1 Prioirty 5 Rule: Min_req - If in last 15 mins if this FT has got min hits of 20 Max_Req - Max can be anything (0 to denote as max can be anything) Failure% - If Failure% is 100 watch - 2 means that I need to check whether in my previous 15min interval (i.e. 10:30-10:45) also this FT is satisfying the Rule 5 or not. If yes then only send the alert otherwise not. Prioirty 4 Rule: Min_req - If in last 15 mins FT has got min hits of 20 Max_Req - Max can be anything (0 to denote as max can be anything) Min_Req_Fail - If Min_Req_Fail is >=301 watch - 1 means that I need not check my prior 15min interval and just send the alert Prioirty 3 Rule: Min_req - If in last 15 mins FT has got min hits of 20 Max_Req - If in last 15 mins FT has got max hits of <=300 Min_Req_Fail - If Min_Req_Fail is >=76 watch - 1 means that I need not check my prior 15min interval and just send the alert Prioirty 2 Rule: Min_req - If in last 15 mins FT has got min hits of 20 Max_Req - If in last 15 mins FT has got max hits of <=300 Min_Req_Fail - If Min_Req_Fail is >=41 Min_Req_Fail - If Min_Req_Fail is <=75 watch - 2 means that I need to check whether in my previous 15min interval (i.e. 10:30-10:45) also this FT is satisfying the Rule 2 or not. If yes then only send the alert otherwise not. Prioirty 1 Rule: Min_req - If in last 15 mins FT has got min hits of 20 Max_Req - If in last 15 mins FT has got max hits of <=300 Min_Req_Fail - If Min_Req_Fail is >=20 Min_Req_Fail - If Min_Req_Fail is <=40 watch - 3 means that I need to check whether in my previous 2 15min interval (i.e. 10:30-10:45, 10:15-10:30) also this FT is satisfying the Rule 1 or not. If yes then only send the alert otherwise not. This is how I need to comapre my log with the rules in the lookup table and generate an alert. Above lookup table is a sample one but it has more rows with diff FT, Error_Code & rules. I am pretty new to splunk and will be a real help if someone can guide me in writing query to achieve this for 1 FT so that I can simulate for entire lookup table. index=dte_fios sourcetype=dte2_Fios FT=*FT earliest=04/20/2020:11:00:00 latest=04/20/2020:13:00:00 | stats count as Total, count(eval(Error_Code!="0000")) AS Failure by FT | eval Failurepercent=round(Failure/Total*100) | table FT, Total,Failure,Failurepercent I need help with how to get the rows for every 15min interval for each FT & ErroCode and then verify the rules ... View more

Hi, I am using below query to get the stats o/p of Total, Failure & Failure percent by couple of fields for every 15 min interval over 2 hrs duration. index=dte_fios sourcetype=dte2_Fios FT=*FT earliest=04/20/2020:11:00:00 latest=04/20/2020:13:00:00 | bin _time span=15m | stats count as Total, count(eval(Error_Code!="0000")) AS Failure by FT,Error_Code,_time | eval Failurepercent=round(Failure/Total*100) I am getting O/p as expected in terms of cols like below: FT Error_Code _time Total Failure Failurepercent ALCATEL_FT 8950 2020-04-20 12:15:00 10 10 100% ALCATEL_FT 8950 2020-04-20 12:30:00 10 5 50% ALCATEL_FT 8950 2020-04-20 12:45:00 10 10 100% The issue here is if any interval is having 0 records (we do not have row for 11, 11:15 11:30 intervals) is is not showing a row. I need the O/P to give row for every 15min interval and show Total & Failure as 0. I tried to use timechart but I could not get the above o/p format as stats is not working with timehcart. Can someone help with the query? ... View more

Hi, I have below scenario where I have query 1 which triggers the condition for my alert to fire if it fires then I want to send the O/P of 2nd query to send in email as tabular data. In that email subject I want to include the time duration of my 2nd query index=dte_fios sourcetype=dte2_Fios FT=*FT Error_Code!=0000 earliest=04/20/2020:11:00:00 latest=04/20/2020:13:00:00 | bin _time span=15m | stats count as Total, count(eval(Error_Code!="0000")) AS Failure by FT,_time | eval Failurepercent=round(Failure/Total*100) | table _time,FT,Total,Failure,Failurepercent | lookup ftthresholdlkp FT | eval alert=case(some condition) | where alert=1 | map search="search index=dte_fios sourcetype=dte2_Fios FT=$FT$ earliest=04/20/2020:12:45:00 latest=04/20/2020:13:00:00 | eval STime=strftime(earliest,"%m/%d %H:%M") , ETime=strftime(latest,"%m/%d %H:%M")| eval AlertType=if($Failurepercent$>50,"RED","AMBER")|table _time,WPID,MGRID,Host,System,DIP_Command,CID,DTE_Command,FT,OSS,Error_Code,Error_Msg" I am trying to send Subject of mail as "AMBER ALERT: Below are the failure from 04/20 12:45 TO 04/20 13:00 GMT GMT" to get I used $result.earliest$ $result.latest$ but they are coming as blank in my subject then I used eval command to create 2 fields STIME & ETime but if I add that in map search it is not returning any rows at all and also my AlertType is also not working Can someone help me out here how can achieve above subject using my query ... View more

Hi, Can someone help with regex expression to mask the below kind of pattern. I need this pattern of text to be masked wherever I find it in my events. 12/KQXA/123456/ABXY --> **************ABXY 11/VAXA/123456 /VAQY --> **************VAQY 00/LCXA/545232/GYFT --> **************GYFT ... View more