We have a number of real time alerts that are working fine (that are being generated by certain Active Directory events via the Universal Forwarder installed on the DC), but when I try to create any new real time alerts they do not seem to work; I am not receiving the email, and the Alert counter on the Searches and Reports page remains on 0. When I run the search manually for the last 15 minutes, I get results that I would expect, so the search parameters seem to be ok.
I even cloned a working rule, and created an event. The original alert triggered, but the new cloned one did not 😞
I am fighting the same problems, but I do have some clues. Mine I beleive are related to LDAP so I don't know your environment but configured to LDAP can be and issue.
My real time alerts changed everytime I changed added more complex strings to LDAP.
I have other ideas about working around this but it takes time.
I did have further problems with this, and I now believe I found the cause.
In my case I think it was simply because I had too many real time searches running, and was hitting my my limit. I believe that you can change the limit in limits.conf as long as your hardware is up to the job. I just cleaned up some stuff, and changed some real time searches/alerts to a daily report and have not had any issues since.
I still get issues with real time alerting every now and then. The last one I had (maybe not exactly the same as this) was resolved by a restart of the splunk services. I would be interested to know if this fixes your problem?