I think I've read this in its entirety 4 times now over the past week. I am having difficulty understanding what the problem is. Let me walk through it and see if writing it down helps... You work...
See more...
I think I've read this in its entirety 4 times now over the past week. I am having difficulty understanding what the problem is. Let me walk through it and see if writing it down helps... You work in IST which is +10.5 hours from CST/DST. You have alert, which the cron schedule says to fire at 1 PM (13:00) in CDT. That's 11:30 PM (23:30) IST. You maybe mistyped "11:00 PM" for that, and maybe that's the issue? Disregarding the 11:00/11:30 issue, the second thing I think you mentioned is that the alert didn't actually come until 11:44, which is a 14 minute delay. The search itself is a lightweight, it should run practically instantly and run-time shouldn't be an issue. The most obvious reason for the 14 minute delay is because your server is too busy at 1 PM CDT to get this out any faster. You should check into that - there's a lot of resources available inside Splunk to see what might be going on, but my guess is just that it's a busy time of the day, coupled with possibly too many "heavy" searches that trigger then. You could also increase the priority of that search, though this doesn't address the core problem and may actually make things *worse* and not better. I mean, maybe better for this one search, and being so fast that's probably OK, but still, it's just trying to hide the bigger problem. Anyway, hope that helps and happy Splunking! -Rich