New Real Time Alerts not working

[yrajah]

Hello,

We have a number of real time alerts that are working fine (that are being generated by certain Active Directory events via the Universal Forwarder installed on the DC), but when I try to create any new real time alerts they do not seem to work; I am not receiving the email, and the Alert counter on the Searches and Reports page remains on 0. When I run the search manually for the last 15 minutes, I get results that I would expect, so the search parameters seem to be ok.

I even cloned a working rule, and created an event. The original alert triggered, but the new cloned one did not 😞

[jkeellogic]

Hey dudes.
I am fighting the same problems, but I do have some clues. Mine I beleive are related to LDAP so I don't know your environment but configured to LDAP can be and issue.
My real time alerts changed everytime I changed added more complex strings to LDAP.

I have other ideas about working around this but it takes time.

jim

[yrajah]

I did have further problems with this, and I now believe I found the cause.

In my case I think it was simply because I had too many real time searches running, and was hitting my my limit. I believe that you can change the limit in limits.conf as long as your hardware is up to the job. I just cleaned up some stuff, and changed some real time searches/alerts to a daily report and have not had any issues since.

http://docs.splunk.com/Documentation/Splunk/5.0.3/Admin/Limitsconf

http://docs.splunk.com/Documentation/Splunk/5.0.1/Search/Realtimeperformanceandlimitations

[salem34]

Thanks for pointing that out, idk if i ever thoght about limits here.

[jtrucks]

Are the newly created/cloned alerts owned by a different user? If so, does that user have a valid email address set?

[jtrucks]

You should post an answer that a reboot fixed it and then accept the answer. 🙂

[brettcave]

nope, owned by my user. i have a valid email address. the alerts started working after a splunk server restart.

[yrajah]

Hello,
I still get issues with real time alerting every now and then. The last one I had (maybe not exactly the same as this) was resolved by a restart of the splunk services. I would be interested to know if this fixes your problem?

[brettcave]

+1 - me too. alerts were working, and then modified the search. now they're not. I even deleted the search, recreated it, and still not getting results.

[sd100]

Hello, Same issue here, i'm interested in the answer.