Splunk ES Email alert and notable events are not getting triggered

[splunky_diamond]

Hello, Splunk community! 

I have created a correlation search with the following search string: 

index="kali2_over_syslog" ((PWD=/etc AND cmd=*shadow) OR (PWD=* cmd=*/etc/shadow)) OR ((PWD=/etc AND cmd=*passwd) OR (PWD=* cmd=*/etc/passwd))
| eval time=strftime(_time, "%D %H:%M") | stats values(host) as "host" values(time) as "access time" values(executer) as "user" count by cmd
| where 'count'>0

When I use it in search and reporting app and executing "sudo cat /etc/shadow" on the monitored linux machine it catches this event. 

The rest of the settings of that correlation search are the same as in my other correlation search, which I used as a template. That another correlation search works well and notable events are getting generated as well as the email notification is sent. 

The only difference is that I am not using any datamodel in my search because I have a small test lab and I only have one machine on which I want to monitor the following activity. Can it be that I must use the CIM-validated data models in my search, so that correlation search actually works fine and generates notable events?

I am new to Splunk, so I am sorry if my question is a bit unclear or weird, let me know if you need additional information 🙂

[kprior201 · Accepted solution]

Hello! You are not required to use data models or CIM compliance in correlation searches, so that isn't the issue here. Just to verify: If you copy/paste this exact search into a regular search bar, it will find results? If that is the case, here's what I would check:

- Is it looking back the same amount of time? Make sure your correlation search is using the same look back as your manual search.

- Are you executing the search within the Enterprise Security app when you are testing? If not, try it there. If the results don't return within the Enterprise Security app but they do within a different app, it could be a permissions/sharing setting for field parsings.

[splunky_diamond]

Thank you very much kprior201! The issue was that I was executing the search not within the ES app when I was testing it, but in the Search and Reporting app. I did not have some of the manually extracted fields in my ES app, once I added them, the correlation search worked well!