All Apps and Add-ons

TA_Azure_Monitor - script running failed (exited with code 1).

morethanyell
Builder

Hi everyone,

An HF node in our env started getting this message all of a sudden.

Unable to initialize modular input "azure_monitor_metrics" defined inside the app "TA_Azure_Monitor": Introspecting scheme=azure_monitor_metrics: script running failed (exited with code 1).

While I understand that the articles below seem to have ended with a solution, I feel like I didn't get any procedure on how to attack this problem and solve it.

These are some of the artifacts I found in our internal logs.

09-25-2019 11:59:33.957 +1000 ERROR ModularInputs - Unable to initialize modular input "azure_monitor_metrics"  defined inside the app "TA_Azure_Monitor": Introspecting scheme=azure_monitor_metrics: script running failed (exited with code 1).
09-25-2019 11:59:33.856 +1000 ERROR ModularInputs - Unable to initialize modular input "azure_diagnostic_logs"  defined inside the app "TA_Azure_Monitor": Introspecting scheme=azure_diagnostic_logs: Unable to run `"/opt/splunk/etc/apps/TA_Azure_Monitor/bin/azure_diagnostic_logs.sh --scheme": child failed to start: Permission denied
09-25-2019 11:59:33.856 +1000 ERROR ModularInputs - Introspecting scheme=azure_diagnostic_logs: Unable to run "/opt/splunk/etc/apps/TA_Azure_Monitor/bin/azure_diagnostic_logs.sh --scheme": child failed to start: Permission denied
09-25-2019 11:59:33.854 +1000 ERROR ModularInputs - Unable to initialize modular input "azure_activity_log"  defined inside the app "TA_Azure_Monitor": Introspecting scheme=azure_activity_log: Unable to run "/opt/splunk/etc/apps/TA_Azure_Monitor/bin/azure_activity_log.sh --scheme": child failed to start: Permission denied
09-25-2019 11:59:33.854 +1000 ERROR ModularInputs - Introspecting scheme=azure_activity_log: Unable to run "/opt/splunk/etc/apps/TA_Azure_Monitor/bin/azure_activity_log.sh --scheme": child failed to start: Permission denied

The box affected is a RHEL.

Thanks in advance.

Article 1. https://github.com/Microsoft/AzureMonitorAddonForSplunk/issues/73
Article 2. https://github.com/Microsoft/AzureMonitorAddonForSplunk/issues/63

0 Karma

ivanreis
Builder

Per the log information you had posted, there is a denied permission to run the child process "child failed to start: Permission denied". This lack of permission can be related with "exit code 1" from the app it self.
Verify if the user that is running splunk service has the proper permission to run this TA app, it is common problem in linux environment for permission issues, but I dont know how the permission works at Azure because I never play with before. I would stop splunk service, re-apply the permissions and start splunk service.

0 Karma

jacobpevans
Motivator

Not a full answer, but try setting ModularInputs component logging to DEBUG and see if you get any more useful information. You can do this through the GUI (Server Settings > Logging)

Cheers,
Jacob

If you feel this response answered your question, please do not forget to mark it as such. If it did not, but you do have the answer, feel free to answer your own post and accept that as the answer.
0 Karma

morethanyell
Builder

Will do. Thanks.

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...