topic Custom alerts logs don't appear in internal index in Getting Data In

Custom alerts logs don't appear in internal index

diletoan — Fri, 29 Mar 2019 14:59:48 GMT

I don't see my custom alert action's logs as the documentation suggests I should.

import sys

# splat
# Run with arbitrary input, e.g., index=_internal | head 1 | sendalert splat

if __name__ == '__main__':
  print >>sys.stderr, "WARN splat look for me in the logs!"
  sys.exit(2)

According to https://docs.splunk.com/Documentation/Splunk/7.2.5/AdvancedDev/ModAlertsLog :

Access alert action script logs
[....] Any information that your script prints to STDERR will be treated as a log message. Message
prefixes, such as DEBUG, INFO, WARN, or ERROR, are treated as the log level. To review logs for an alert
action, select Settings>Alert actions.
This takes you to the Alert Actions manager page. Select View log events for your alert action.

When I run the above custom alert, I see nothing in the internal index. I do see its logs in search.log if it exits non-zero, of course, but I'd like to be able to see them from the View log events link.

How can I see that WARN log line in View log events (viz., index=_internal sourcetype=splunkd component=sendmodalert action="splat") as the documentation suggests I ought to?

Re: Custom alerts logs don't appear in internal index

harsmarvania57 — Mon, 01 Apr 2019 08:21:03 GMT

Hi,

Instead of if name == 'main':, can you please run simple script without that if condition ? Have a look at sample example script on doc https://docs.splunk.com/Documentation/Splunk/7.2.5/AdvancedDev/ModAlertsBasicExample

Re: Custom alerts logs don't appear in internal index

diletoan — Mon, 01 Apr 2019 11:59:34 GMT

@harsmarvania57 , thanks. 'main' was a formatting error of course. I could not persuade the indented code block to be formatted correctly, so I moved it. See again. I can try your simpler example, but it's not germane: I know that the script runs, and that if stanza is good python and in most other splunk docs (e.g., the HipChat handler in those docs). I just don't know where the output goes.

Re: Custom alerts logs don't appear in internal index

diletoan — Mon, 01 Apr 2019 15:44:09 GMT

The explanation appears to be that the stderr logs are properly captured to the internal index when a bona fide alert is configured (Searches, Reports, and Alerts).

However, they are not captured when the same custom alert is run by hand using sendalert my_custom_alert.

That's unexpected behavior, I'd say, but so it is.