I tried the first query and it did not produce the expected results. I tried timechart but it does not seem to play well with the Outlier query in Splunk Docs and I am just starting in Splunk so I could not get it to work.
I don't think the one second and one minute interval will work; the servers are load balanced but at that rate it would produce hits one even small changes; although you are right 1 hour is a bit long. I was mainly trying to get it to work then I can fine tune the intervals. I did try the queries to see what would happen and I had hits even during periods when all servers on the farm were responding.
I did try the deviations but I could not get them to work and stumbled on the Outlier example in the Splunk Docs.
I will look at your chart examples; I am just getting started so I am not sure if I am up to that challenge yet, Thanks for the tips and examples they helped and provided a better query than what I can up with.
P.S. the sort was more to put hits first when I am reviewing results - you make a good point that it is not needed.
... View more