Alerting

ITSI Custom Alert Action issue

shaan
Engager

We have created the custom Action in ITSI to POST the Episode data to outside world.

Steps taken for creation of Custom action:

  1. Created App
  2. Created Python Script in Bin Folder of application
  3. Alert_action.conf creation inside the app
  4. Added <AppName> stanza in notable_event_action.conf in ITSI SA-ITOA Folder

After this the Action name is visible in Action drop down menu of ITSI Episode.

Issue:

Whenever we are selecting the custom action it is throwing error.

Action = "AppNme" failed to run. Refer logs for more information.

When we refer the appserver.log it is giving us the below error

Can anyone please help on this, How to deal with this kind of error ? It seems Action itself is not getting called from ITSI what are your views ?

Error log-
=============================================================
Status="Started", FunctionName="actionExecution", actionInternalName="iPaaS"
2020-06-24 17:01:47,601 INFO [appserver] [notable_event_actions] [get_event_ids] [12252] is_group=True. action_name=`iPaaS` received=`[u'90cb2e34-08f4-463b-a395-4ce4a383e198']`

2020-06-24 17:01:47,603 INFO [appserver] [notable_event_actions] [execute_action] [12252] Generated search command=`search `itsi_event_management_group_index
` itsi_group_id="90cb2e34-08f4-463b-a395-4ce4a383e198" | dedup itsi_group_id | fields * | `itsi_notable_event_actions_temp_state_values` | `itsi_notable_group_lookup` | `itsi_notable_event_actions_coalesce_state_values` | sendalert "iPaaS" param.url="https://external_tool.com "` for action=`iPaaS` with earliest_time=None, latest_time=None

2020-06-24 17:01:48,303 ERROR [appserver] [notable_event_actions] [_parse_and_call_execute_action] [12252] Actions (iPaaS) failed to execute on: [u'90cb2e34-
08f4-463b-a395-4ce4a383e198']. Error: search `itsi_event_management_group_index` itsi_group_id="90cb2e34-08f4-463b-a395-4ce4a383e198" | dedup itsi_group_id
| fields * | `itsi_notable_event_actions_temp_state_values` | `itsi_notable_group_lookup` | `itsi_notable_event_actions_coalesce_state_values` | sendalert
"iPaaS" param.url="https://external_tool.com "  search failed. Refer search.log at "/services/search/jobs/1593014507.15717/search.log".

2020-06-24 17:01:48,303 INFO [appserver] [notable_event_actions] [_parse_and_call_execute_action] [12252] actionId="None", actionName="notable_event_action",
Status="Failed", FunctionName="actionExecution
===============================================================

Labels (2)

Oliver
Engager

Can you check Search logs mentioned into the error,  could be permissions on the  app to ensure that they are set to Global or App instead of Private.  Also, could there be some restriction on the user you are accessing Splunk ?

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...