Alerting

How to schedule search and alert for Windows services that are "Stopped", set to "Auto" and system uptime is < 86400 seconds

agoktas
Communicator

Hello,

I'm hoping someone can help me assemble this search & alert.

We have two indexes:

index_evt is for windows event logs
index_perf contains all performance metrics & windows service information

I want to create a search that
- Searches index_perf (source=WMI:Services) for any service that contains "IBM" in the service name, that is "Stopped" and is also set to "Auto" for startup type within the past 2 hours.
- Searches index_evt (sourcetype="WinEventLog:System") for system uptime is < 86400 seconds.

We're looking for windows services that should be running after reboots of windows servers.

Your help is truly appreciated.

Thanks.

0 Karma

lguinn2
Legend

It would help to have the field names for these sources, but I will just make up field names...

index=index_evt sourcetype="WinEventLog:System" uptime >=1800 uptime<=86400
| join host max=0 [ search index=index_perf source="WMI:Services" status=Stopped startup=Auto
                    | where match(service_name, "IBM") ]

Hopefully this is a good starting point...

0 Karma

lguinn2
Legend

First, run the two searches separately? Do you get results? Do the results have a field named host? Does the host field have the same format, etc. and does it match across the two searches?

Finally, is it possible that 0 results is the right answer?

0 Karma

agoktas
Communicator

Hi Iquinn,

Sorry for the late reply.

I have done a search and successful results with the following two searches (separating both searches to verify they each work):

- index=app_evtlogs sourcetype="WinEventLog:System" | rex "The system uptime is (?\d+) seconds." | Where uptime<=86400
- index=app_perf source="WMI:Services" State=Stopped StartMode=Auto | rex field=_raw "\nName=(?PIBM.{1,})" | top SvcName by host

But when I try to do a join:
index=app_evtlogs sourcetype="WinEventLog:System" | rex "The system uptime is (?\d+) seconds." | Where uptime<=86400 | join host max=0 [ index=app_perf source="WMI:Services" State=Stopped StartMode=Auto | rex field=_raw "\nName=(?PIBM.{1,})" | top SvcName by host ]

I get:
Unknown search command 'index'.

Any ideas as to what I'm missing or doing wrong?

Am I even able to do a 'join' with each search referencing a different index?

Thanks!

0 Karma

agoktas
Communicator

Thanks for your quick response lguinn! 🙂

I get 0 results back.

One thing I had to do was:
index=index_evt sourcetype="WinEventLog:System" | rex "The system uptime is (?\d+) seconds." | Where uptime >=0 AND uptime<=86400

But when appending the rest, I get nothing.

Full search:

index=index=evt sourcetype="WinEventLog:System" | rex "The system uptime is (?\d+) seconds." | Where uptime >=0 AND uptime<=86400 | join host max=0 [ search index=index_perf source="WMI:Services" status=Stopped startup=Auto | where match(service_name, "IBM") ]

Any other ideas?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...