https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/374001#M6632 <P>thank you micahkemp, it worked like charm.</P> Tue, 13 Feb 2018 20:34:14 GMT newbie2tech 2018-02-13T20:34:14Z https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/373999#M6630 <P><CODE>enter code here</CODE>Hi Team,</P> <P>We have an application platform which is gateway for multiple applications[1000+ apps], we have ingested platform logs in splunk.</P> <P>Platform team wants to setup alerts for each of the applications if the number of HTTP 500 requests breaches the threshold say 5% of the total requests for that app. Alert schedule is every 5 mins. Alert should be sent to corresponding app team as there are more than 1000+ apps, platform team cannot handle.</P> <P>I am using the top command with limit=0 , this is giving the total events across all the applications in last 5 mins but i want to be able treat the count of specific application as 100 and then calculate the 500 requests % and check if the threshold is violated.</P> <P>my current query</P> <PRE><CODE>index=abc | top HttpStatus app limit=0 </CODE></PRE> <P>This is how my current query data looks like , App1+App2 adds up to 100%</P> <PRE><CODE>HttpStatus app count percent 400 App1 30 0.091609 200 App1 15 0.045804 500 App1 6 0.018322 200 App2 3813 11.643459 400 App2 2 0.006107 500 App2 28882 88.194699 </CODE></PRE> <P>This is how i want the output to be able to trigger at app level. app1 breakup sums up to 100%,likewise App2 breakup sums up to 100%. Then i can link it to lookup and check if the 500 requests have exceeded the threshold % and then trigger the alert.</P> <PRE><CODE>HttpStatus app count percent 400 App1 30 58.82352941 200 App1 15 29.41176471 500 App1 6 11.76470588 200 App2 3813 11.66162033 400 App2 2 0.006116769 500 App2 28882 88.3322629 </CODE></PRE> <P>I want to use single consolidated alert which can trigger for each applications threshold violation[i plan to use lookup for thresholds] as we do not want to set up 1000 odd individual alerts specific to application</P> <P>Can you share your thoughts if there is a way to achieve this or something similar.</P> <P>Thanks!</P> Tue, 13 Feb 2018 03:50:51 GMT https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/373999#M6630 newbie2tech 2018-02-13T03:50:51Z https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/374000#M6631 <P>Try:</P> <PRE><CODE>index=abc | top HttpStatus app limit=0 | eventstats sum(count) AS app_total BY app | eval app_percent=(count/app_total)*100 </CODE></PRE> Tue, 13 Feb 2018 03:56:25 GMT https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/374000#M6631 micahkemp 2018-02-13T03:56:25Z https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/374001#M6632 <P>thank you micahkemp, it worked like charm.</P> Tue, 13 Feb 2018 20:34:14 GMT https://community.splunk.com/t5/Alerting/Consolidated-Alerting-using-TOP-command/m-p/374001#M6632 newbie2tech 2018-02-13T20:34:14Z