Reporting

How do you manually add entries to savedsearches.conf for alerts?

reallyliri
Explorer

I want to create a lot of saved searches for alerts. Because I need to create about 20 different ones, I prefer to do it programatically.

I wrote a short program to generate the .conf file and replaced it with the existing one. However, after that, no alerts were triggered at all.

I checked again and again, and the the entries look like the splunk-generated ones.

From that, I'm assuming that the problem is that the file shouldn't be edited manually, or that there is some additional editing that needs to be done?

Any help is much appreciated.

Example for a splunk-generated entry:

[Errors alert]
action.email = 1
action.email.include.trigger_time = 1
action.email.inline = 1
action.email.mailserver = localhost
action.email.sendresults = 1
action.email.to = mymail@gmail.com
action.email.useNSSubject = 1
action.slack_webhook_alert = 1
action.slack_webhook_alert.param.message = Your *$name$* alert matched at least $job.resultCount$ events. Link: $results_link$. First result:    ```$result.exc_info$```
action.slack_webhook_alert.param.slack_webhook_name = Slack-Alerts
alert.suppress = 1
alert.suppress.period = 60s
alert.track = 1
counttype = number of events
cron_schedule = * * * * *
description = logs with level > INFO
dispatch.earliest_time = rt-1m
dispatch.latest_time = rt-0m
enableSched = 1
quantity = 0
relation = greater than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = source="*-server" host=dev_*| spath levelno | search levelno>20

Example for a progrmatically-generated entry:

[Errors alert]
action.email = 1
action.email.include.trigger_time = 1
action.email.inline = 1
action.email.mailserver = localhost
action.email.priority = 2
action.email.sendresults = 1
action.email.to = mymail@gmail.com
action.email.useNSSubject = 1
action.slack_webhook_alert = 1
action.slack_webhook_alert.param.message = Your *$name$* alert matched at least $job.resultCount$ events. Link: $results_link$. First result:    ```$result._raw$```
action.slack_webhook_alert.param.slack_webhook_name = Slack-Alerts
alert.suppress = 1
alert.suppress.period = 60s
alert.track = 1
counttype = number of events
cron_schedule = * * * * *
description = logs with level > INFO
dispatch.earliest_time = rt-1m
dispatch.latest_time = rt-0m
enableSched = 1
quantity = 0
relation = greater than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = source="*-server" host=dev_*| spath levelno | search levelno>20
0 Karma

ddrillic
Ultra Champion

-- From that, I'm assuming that the problem is that the file shouldn't be edited manually, or that there is some additional editing that needs to be done?

Any config file can be edited manually. Probably _internal would have some information ...

0 Karma

reallyliri
Explorer

Best I could do was add the saved-search with the cli, then manually edit the savedsearched.config file, but surely theres a simpler way. The documentation is horrible.

/opt/splunk/bin/splunk add saved-search -name 'Errors' -search 'source="*-server" host=dev_*| spath levelno | search levelno>20'
0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...