Monitoring Splunk

List correlation searches which did not trigger a notable in past X days

fedejko
Explorer

Hi,

I have a list of all notable events which triggered in X days using this SPL:

 

index=notable search_name="*Rule" orig_action_name=notable
| stats count by search_name

 

 

Using this query I can see the list of my all rules which are enabled to trigger notables: 

 

| rest /services/saved/searches
| search title="*Rule" action.notable=1
| table title

 

Obviusly, the second search returns much larger list. I'd like to correlate those two searches to find out which of all the rules did not dispatch a notable in past X days.

Any ideas on how to achieve this?

Labels (2)
0 Karma

fedejko
Explorer

Thanks, I was thinking about the join, too, but finally I came up with something like this:

| rest /services/saved/searches
| search title="*Rule" action.notable=1
| rename title as rule_name
| search NOT
[ search index=notable search_name="*Rule" orig_action_name=notable
| stats count by search_name
| rename search_name as rule_name
| table rule_name ]
| table rule_name ]

I thought it would be faster than joins.

0 Karma

ccl0utier
Splunk Employee
Splunk Employee

@fedejko You should be able to join the two sets together, like this, for example (reusing your two queries):

| rest /services/saved/searches
| search title="*Rule" action.notable=1
| fields title
| eval has_triggered_notables = "false"
| join type=outer title [
   search index=notable search_name="*Rule" orig_action_name=notable
   | stats count by search_name
   | fields - count
   | rename search_name as title
   | eval has_triggered_notables = "true"
]

That adds a field has_triggered_notables which will indicate if a rule has triggered notable and you can then filter out the results to your liking, for example, to show only the rules which did trigger notables:

| where has_triggered_notables = "true"
Get Updates on the Splunk Community!

Splunk Smartness with Brandon Sternfield | Episode 3

Hello and welcome to another episode of "Splunk Smartness," the interview series where we explore the power of ...

Monitoring Postgres with OpenTelemetry

Behind every business-critical application, you’ll find databases. These behind-the-scenes stores power ...

Mastering Synthetic Browser Testing: Pro Tips to Keep Your Web App Running Smoothly

To start, if you're new to synthetic monitoring, I recommend exploring this synthetic monitoring overview. In ...