Alerting

Why is the custom alert script failing with sendalert?

jrehl01
New Member

I'm trying to create a custom alert application. All I want to do right now is to see what kind of parameters I can pull and utilize. So I'm just doing a simple print into a file. I created an app called say, test. And the script is called showconfiguration. This is my first time making a custom app.

Here's what the directory structure looks like under /opt/splunk/etc/apps/test:
test/

├── appserver
│   └── static
│       └── icon.png
├── bin
│   └── showconfiguration.py
├── default
│   ├── alert_actions.conf
│   ├── app.conf
│   └── data
│       └── ui
│           └── alerts
│               └── showconfiguration.html
├── metadata
│   ├── default.meta
│   └── local.meta
└── README
    └── alert_actions.conf.spec

Here's alert_actions.conf contents:
> [showconfiguration]
>is_custom = 1
>label = Testing Splunk alerting capability
>description = Testing
>icon_path = icon.png
>payload_format = json
>param.trigger_reason = Saved Search [test] number of events ($job.resultCount$)
>param.result_count = $job.resultCount$
>param.one = two


Contents of app.conf:
> [ui]
>is_visible = 0
>label = Alert Tests
>
>[launcher]
>author = Me
>description = Testing splunk alert capability
>version = 0.1
>
>[install]
>state = enabled
>is_configured = 1

And everything is owned by splunk:splunk and I think has the correct permissions.
So I create an alert and set it to run this custom alert. It never runs and I see this error in the splunkd.log:

08-29-2018 15:10:40.746 -0400 ERROR sendmodalert - Error in 'sendalert' command: Alert action "showconfiguration" not found.

I don't know what I did wrong here. How do I get sendalert to recognize the showconfiguration script??

Labels (2)
0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Try below script and do not include #!/usr/bin/python in your script because splunk will use inbuilt python.

import pprint, json, sys

if __name__ == "__main__":
  if len(sys.argv) > 1 and sys.argv[1] = "--execute":
    f.open("/tmp/splunktest.txt", "w")
    f.write("Here's the info we get from splunk:")
    f.write(pprint.pprint(json.loads(sys.stdin.read())))
    f.close()

View solution in original post

0 Karma

jrehl01
New Member

Here is the end result of the script that pretty much shows all the information you can grab about an alert and the corresponding events:

import json, sys, csv, gzip

if __name__ == "__main__":
     if len(sys.argv) > 1 and sys.argv[1] == "--execute":
        data = json.loads(sys.stdin.read())
        f = open("/tmp/splunktest.txt", "w")
        f.write("Here's the info we get from splunk:\n")
        f.write(json.dumps((data), indent=4, sort_keys=False))
        f.write("\n\nResults Data:\n")
        results_file = data["results_file"]

        fz = gzip.open(results_file)
        results_content = csv.DictReader(fz)
        for idx, row in enumerate(results_content):
            f.write("Information for result #" + str(idx) + "\n")
            for key, value in row.iteritems():
                f.write("Key: " + str(key) + "\tValue: " + str(value) + "\n")
            f.write("\n")
        fz.close()
        f.close()
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Try below script and do not include #!/usr/bin/python in your script because splunk will use inbuilt python.

import pprint, json, sys

if __name__ == "__main__":
  if len(sys.argv) > 1 and sys.argv[1] = "--execute":
    f.open("/tmp/splunktest.txt", "w")
    f.write("Here's the info we get from splunk:")
    f.write(pprint.pprint(json.loads(sys.stdin.read())))
    f.close()
0 Karma

pjohnson1
Path Finder

I'm following this example but get the following error. Any idea of what I did wrong?

04-02-2020 07:30:11.193 +0000 WARN  sendmodalert - action=showconfiguration - Alert action script returned error code=1
04-02-2020 07:30:11.193 +0000 INFO  sendmodalert - action=showconfiguration - Alert action script completed in duration=11 ms with exit code=1
04-02-2020 07:30:11.191 +0000 ERROR sendmodalert - action=showconfiguration STDERR -  SyntaxError: invalid syntax
04-02-2020 07:30:11.191 +0000 ERROR sendmodalert - action=showconfiguration STDERR -                                           ^
04-02-2020 07:30:11.191 +0000 ERROR sendmodalert - action=showconfiguration STDERR -      if len(sys.argv) > 1 and sys.argv[1] = "--execute":
04-02-2020 07:30:11.191 +0000 ERROR sendmodalert - action=showconfiguration STDERR -    File "/opt/splunk/etc/apps/showconfiguration/bin/showconfiguration.py", line 4
host = 6c83f2e55cd4source = /opt/splunk/var/log/splunk/splunkd.logsourcetype = splunkd
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Please open a new question and refer this question link in new question, also provide your script in new question.

0 Karma

pjohnson1
Path Finder

Thanks Harshil.

0 Karma

jrehl01
New Member

It was more the metadata/default.meta file that was the issue, but I'll still accept you as providing the answer. Thanks again!

0 Karma

jrehl01
New Member

Unfortunately still didn't help.
Here's the logs again:

08-30-2018 08:05:32.379 -0400 INFO sendmodalert - Invoking modular alert action=showconfiguration for search="Test" sid="rt_scheduler_adminsearch_RMD5742b25d78d6cf18c_at_1535567638_0.185" in app="search" owner="admin" type="saved"

08-30-2018 08:05:32.380 -0400 ERROR sendmodalert - Error in 'sendalert' command: Alert action "showconfiguration" not found.

08-30-2018 08:05:32.380 -0400 ERROR SearchScheduler - Error in 'sendalert' command: Alert action "showconfiguration" not found., search='sendalert showconfiguration results_file="/opt/splunk/var/run/splunk/dispatch/rt_scheduler_adminsearchRMD5742b25d78d6cf18c_at_1535567638_0.185/results.csv.gz" results_link="http://s01-splunk-d01.devmss.leidos.com:8000/app/search/@go?sid=rt_scheduleradminsearch_RMD5742b25d78d6cf18c_at_1535567638_0.185"'

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

After changing configuration have you restarted splunk ? If not then please restart splunk.

0 Karma

jrehl01
New Member

I just restarted and same issue.
However, I'm noticing this piece in the log:

Unable to find alert action script for action="showconfiguration" in app="system"

Why is it looking in the "System" app and not my own? Do I have do update anything in the /opt/splunk/etc/system directory?

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Do you have below configuration in $SPLUNK_HOME/etc/apps/test/metadata/default.meta, if not then please do below settings, restart splunk and try again.

[]
access = read : [ * ], write : [ admin ]

[alert_actions/showconfiguration]
export = system
access = read : [ * ], write : [ admin ]

[alerts]
export = system

jrehl01
New Member

That was it! I fixed some errors in the script and it successfully ran.
You've been very helpful, thank you!

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Great, I have converted my comment to answer. So that you can accept/upvote it. Also it will be good to provide script which you modified so that other community member will able to use that script in future (Please remove any sensitive data).

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @jrehl01,

showconfiguration.html should be in $SPLUNK_HOME/etc/apps/test/data/ui/alerts/, also it will be good if you will provide content of your python script.

0 Karma

jrehl01
New Member

It is in that directory. For some reason the spacing that I put in the post isn't reflected when I submitted it.
Here's the content of the script:

#!/usr/bin/python

import pprint, json, sys

if name == "main":
f.open("/tmp/splunktest.txt", "w")
f.write("Here's the info we get from splunk:")
f.write(pprint.pprint(json.loads(sys.stdin.read())))
f.close()

0 Karma

jrehl01
New Member

Oh I didn't even notice the formatting was off on the tree. It is under that directory, the formatting of this post just doesn't reflect that. Not sure why I can't update the amount of spacing there.

Here's the content of the script:

!/usr/bin/python

import pprint, json, sys

if __name__ == "__main__":
f.open("/tmp/splunktest.txt", "w")
f.write("Here's the info we get from splunk:")
f.write(pprint.pprint(json.loads(sys.stdin.read())))
f.close()

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...