I want to implement automatic mail alerts triggered for the client applications which invoke our application through web services by get/post requests, in order to find a high number of requests incoming from an application or slow requests caused by an application
I also want to reduce the manual efforts by searching each and every index of calls and by getting the client hostname by xforwarded and by DNS lookup in Splunk.
The goal is to be more proactive about planning for this. Please help me out.
Since you have searches already (the ones you run manually), you can save them as alerts that are triggered by non-zero results and have them send email.