I want to create a lot of saved searches for alerts. Because I need to create about 20 different ones, I prefer to do it programatically.
I wrote a short program to generate the .conf file and replaced it with the existing one. However, after that, no alerts were triggered at all.
I checked again and again, and the the entries look like the splunk-generated ones.
From that, I'm assuming that the problem is that the file shouldn't be edited manually, or that there is some additional editing that needs to be done?
Any help is much appreciated.
Example for a splunk-generated entry:
[Errors alert]
action.email = 1
action.email.include.trigger_time = 1
action.email.inline = 1
action.email.mailserver = localhost
action.email.sendresults = 1
action.email.to = mymail@gmail.com
action.email.useNSSubject = 1
action.slack_webhook_alert = 1
action.slack_webhook_alert.param.message = Your *$name$* alert matched at least $job.resultCount$ events. Link: $results_link$. First result: ```$result.exc_info$```
action.slack_webhook_alert.param.slack_webhook_name = Slack-Alerts
alert.suppress = 1
alert.suppress.period = 60s
alert.track = 1
counttype = number of events
cron_schedule = * * * * *
description = logs with level > INFO
dispatch.earliest_time = rt-1m
dispatch.latest_time = rt-0m
enableSched = 1
quantity = 0
relation = greater than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = source="*-server" host=dev_*| spath levelno | search levelno>20
Example for a progrmatically-generated entry:
[Errors alert]
action.email = 1
action.email.include.trigger_time = 1
action.email.inline = 1
action.email.mailserver = localhost
action.email.priority = 2
action.email.sendresults = 1
action.email.to = mymail@gmail.com
action.email.useNSSubject = 1
action.slack_webhook_alert = 1
action.slack_webhook_alert.param.message = Your *$name$* alert matched at least $job.resultCount$ events. Link: $results_link$. First result: ```$result._raw$```
action.slack_webhook_alert.param.slack_webhook_name = Slack-Alerts
alert.suppress = 1
alert.suppress.period = 60s
alert.track = 1
counttype = number of events
cron_schedule = * * * * *
description = logs with level > INFO
dispatch.earliest_time = rt-1m
dispatch.latest_time = rt-0m
enableSched = 1
quantity = 0
relation = greater than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = source="*-server" host=dev_*| spath levelno | search levelno>20
-- From that, I'm assuming that the problem is that the file shouldn't be edited manually, or that there is some additional editing that needs to be done?
Any config file can be edited manually. Probably _internal
would have some information ...
Best I could do was add the saved-search with the cli, then manually edit the savedsearched.config file, but surely theres a simpler way. The documentation is horrible.
/opt/splunk/bin/splunk add saved-search -name 'Errors' -search 'source="*-server" host=dev_*| spath levelno | search levelno>20'