Alerting

Need to trigger an alert when a process is not running

ajromero
Path Finder
Need to trigger an alert when a process is not running, here is my query but I can not the alert to work index="os" source="Perfmon:Process" host="vm* process_name="RT*" | dedup host process | join host [search index="os" source=Perfmon:Process host="vm*" process_name="RT*" | stats latest(host) latest(_time) by host |eval lastSeen='latest(_time)'|fields host lastSeen] |eval status=if(lastSeen<(_time - 300), "not running","running") |table host status process_name | search status = "not running" or is there another way to look for windows process not running thank you
Tags (3)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

Your search is a little odd - there is no need to join the first search with the second as they are searching the same data

index="os" source="Perfmon:Process" host="vm*" process_name="RT*" 
| stats latest(_time) as lastSeen by host process_name
| where lastSeen>=relative_time(now(), "-300s")
| eval status="not running"
| table host status process_name 

Does that work for you? Are you looking to get the latest HOST time or process_name time?

It's splits by process_name, but you may then get different times for each process. If you want to find if it's the HOST that's not running, then you could add

| eventstats max(lastSeen) as lastSeenHost by host

and change the where clause to use lastSeenHost in the test.

However, if your host does not report ANY data in your search time window, you can only know that it's not running if you know the name of all hosts that _SHOULD_ be running, which you would do by populating a lookup with all expected hosts and then using that list to validate whether the host is present or not

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

Your search is a little odd - there is no need to join the first search with the second as they are searching the same data

index="os" source="Perfmon:Process" host="vm*" process_name="RT*" 
| stats latest(_time) as lastSeen by host process_name
| where lastSeen>=relative_time(now(), "-300s")
| eval status="not running"
| table host status process_name 

Does that work for you? Are you looking to get the latest HOST time or process_name time?

It's splits by process_name, but you may then get different times for each process. If you want to find if it's the HOST that's not running, then you could add

| eventstats max(lastSeen) as lastSeenHost by host

and change the where clause to use lastSeenHost in the test.

However, if your host does not report ANY data in your search time window, you can only know that it's not running if you know the name of all hosts that _SHOULD_ be running, which you would do by populating a lookup with all expected hosts and then using that list to validate whether the host is present or not

 

Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...