I have tried to create a Modular Alert, running under Splunk 7.6.2, based on the blog by Luke Murphy: https://www.splunk.com/blog/2016/08/22/how-to-create-a-modular-alert.html.
I thought I had thoroughly followed the instructions and indeed I can see in the search results where the custom action appears to run successfully:
10/28/19 10-28-2019 12:10:00.649 -0400 INFO sendmodalert - action=make_a_log_message - Alert action script completed in duration=332 ms with exit code=0
10/28/19 10-28-2019 12:10:00.315 -0400 INFO sendmodalert - Invoking modular alert action=make_a_log_message for search="Modular Alert" sid="scheduler_adminsearch_RMD5a82eb824001bf163_at_1572279000.1 " in app
However, none of the other expected logging is happening.
None of the troubleshooting techniques I could think of as well as read about, have provided any insights to what I am doing wrong...
Any help anyone can provide as to how to troubleshoot, would be greatly appreciated.
None of the print statements I put in, no matter where they were, generated any output at all.
The only time I saw any output was when I changed the format of the print statement.
So, I ended up attempting to run the Python script standalone:
I'm running under Debian Linux so the command looked like this:
sudo bin/splunk cmd python etc/apps/modular_alert_example/bin/make_a_log_message.py
and quickly realized that the Python script I was running was written for Python 3. However, my environment is Python 2.7.13.
So, I left the directory structure, and everything else in place and simply replaced the make_a_log_message.py script with a simple "Hello World" script written to a file.
And that was successful!
He is a copy of the test script (for anyone else who might have a need for something like this)
from future import print_function
import sys, os, datetime
f = open(os.path.join("/opt/splunk", "var", "log", "splunk", "test_modalert.log"), "a")
print(str(datetime.datetime.now().isoformat()), "Hello World", file=f)
and here is a copy of the test file contents:
2019-10-29T14:38:49.213920 Hello World
2019-10-29T14:52:00.973576 Hello World