Alerting

How to write search that match Alerts vs Reports on Splunk savedsearch/reports/alerts page?

VatsalJagani
SplunkTrust
SplunkTrust

Alerts vs Reports on Splunk "Searches, reports and alerts" page

 

I want to make this query to show the number of alerts and number of reports that match exactly how it shows on the "Searches, reports and alerts" page.

 

 

| rest /servicesNS/-/-/saved/searches
<eval for type here>
| stats count by type

 

 

 

I found this question long ago but no answer given to an exact matching number of count - https://community.splunk.com/t5/Alerting/What-is-the-difference-between-alert-and-report/m-p/368683 

Woodcock mentioned this, which is a nice explanation of why there is no difference between alert and report anymore.

 

 

Originally only alerts had alert actions but customers insisted and now reports also can have alert actions so literally there is no functional difference between the two. There is now only a taxonomical difference which you are free to slice any way that you like. Settings-wise, the difference between the two now is defined in savedsearches.conf as: alert.track=1 means alert and alert.track=0 means report. That is it.

 

 

 

The main thing is I want to find out how Splunk is deciding whether it's alert or report on the web?

Labels (1)
0 Karma
1 Solution

VatsalJagani
SplunkTrust
SplunkTrust

After doing so many trials and errors and comparison between the Splunk web page and the rest endpoint results here is an eval statement that I made.

| rest /servicesNS/-/-/saved/searches splunk_server=local 
| rename "alert.track" as alert_track
| eval type=case(alert_track=1, "alert", (isnotnull(actions) AND actions!="") AND (isnotnull(alert_threshold) AND alert_threshold!=""), "alert", (isnotnull(alert_comparator) AND alert_comparator!="") AND (isnotnull(alert_type) AND alert_type!="always"), "alert", 1==1, "report")
| stats count by type
| addcoltotals

This query matches reports (353) and alerts (1082) exactly with the Splunk web page for my environment.

 

This is by no means the final query, but it's more of an open-ended answer if someone finds it's not matching in your environment, please add a comment with your findings so we can improve the query.

With my analysis I looked following parameters in rest output for comparison:

  • actions
  • alert_comparator
  • alert_threshold
  • alert.track
  • alert_type

View solution in original post

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

After doing so many trials and errors and comparison between the Splunk web page and the rest endpoint results here is an eval statement that I made.

| rest /servicesNS/-/-/saved/searches splunk_server=local 
| rename "alert.track" as alert_track
| eval type=case(alert_track=1, "alert", (isnotnull(actions) AND actions!="") AND (isnotnull(alert_threshold) AND alert_threshold!=""), "alert", (isnotnull(alert_comparator) AND alert_comparator!="") AND (isnotnull(alert_type) AND alert_type!="always"), "alert", 1==1, "report")
| stats count by type
| addcoltotals

This query matches reports (353) and alerts (1082) exactly with the Splunk web page for my environment.

 

This is by no means the final query, but it's more of an open-ended answer if someone finds it's not matching in your environment, please add a comment with your findings so we can improve the query.

With my analysis I looked following parameters in rest output for comparison:

  • actions
  • alert_comparator
  • alert_threshold
  • alert.track
  • alert_type
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @VatsalJagani,

I can confirm the answer of @woodcock, the difference between a scheduled report and an email is only conceptual, not structural and the only ways to identify the type is the parameter alert.track.

About your final questions: "The main thing is I want to find out how Splunk is deciding whether it's alert or report on the web?"

for Spluk every object is a search (alert, report, dashboard panel, etc...) and each search generates a result and sometimes an action, this means that Splunk doesn't decide if a search is an alert or a report, it executes a search and executes the eventual related actions.

What is is your problem?

If you want to identify the kind of a savedsearch, you can use the alert.track parameter or take the use of insert in the title "Report" or "Alert" (I usually do this!), but there isn't any difference in execution and use.

Ciao.

Giuseppe

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@gcusello - Somehow when I'm trying to execute the below query it does not match the count of reports and alerts that Splunk shows on the web page.

| rest /servicesNS/-/-/saved/searches splunk_server=local
| rename "alert.track" as alert_track
| eval type=if(alert_track=1, "alert", "report")
| stats count by type
| addcoltotals

 

The total is exactly the same 1435 in my case.

The query shows only 217 alerts out of it, while "Searches, Reports, and Alerts" page shows 1082 as alerts and rest reports.

 

My goal is to match what shows under "Searches, Reports, and Alerts" page with the query.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @VatsalJagani,

I don't know why this happen it's difficoult without accessing your system.

Anyway, try to filter results by App to understand if the difference is related to something internal of Splunk, maybe the difference isn'r relevant for your reporting.

Ciao.

Giuseppe

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@gcusello - It is showing the correct count with the app that I developed.

For example, ES Content Update App shows all as Reports with a query, but the Splunk, web page shows 852 as alerts.

After lots of try and error, I've created an eval, putting it in a separate answer.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...