All Apps and Add-ons

Unable to initialize modular input "mscs_storage_table" defined in the app "Splunk_TA_microsoft-cloudservices"

lukasmecir
Path Finder

Hello,
I have a problem with Splunk Add-on for Microsoft Cloud Services. It is installed on Heavy Forwarder in a distributed environment and is used to get data from MS Azure. Everything was fine till yesterday - after a restart of Heavy Forwarder I got this message:

"Unable to initialize modular input "mscs_storage_table" defined in the app "Splunk_TA_microsoft-cloudservices": Introspecting scheme=mscs_storage_table: script running failed (exited with code 1).."

I ran command:

splunk cmd splunkd print-modinput-config --debug mscs_storage_table

and got this output (just interesting part):

Found scheme="mscs_storage_table".
Locating script for scheme="mscs_storage_table"...
No regular file="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.sh".
No regular file="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.py".
No regular file="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.js".
No regular file="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table".
No script found in dir="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin"
No regular file="/opt/splunk/etc/slave-apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.sh".
No regular file="/opt/splunk/etc/slave-apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.py".
No regular file="/opt/splunk/etc/slave-apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table.js".
No regular file="/opt/splunk/etc/slave-apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin/mscs_storage_table".
No script found in dir="/opt/splunk/etc/slave-apps/Splunk_TA_microsoft-cloudservices/linux_x86_64/bin"
No regular file="/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_table.sh".
Found script "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_table.py" to handle scheme "mscs_storage_table".
Ignoring parameter "index" from the spec file: it is an internal arg that gets handled in a special way.
Ignoring parameter "sourcetype" from the spec file: it is an internal arg that gets handled in a special way.
stderr Introspecting scheme=mscs_storage_table:  Traceback (most recent call last):
stderr Introspecting scheme=mscs_storage_table:    File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_table.py", line 7, in module
stderr Introspecting scheme=mscs_storage_table:      import splunktaucclib.data_collection.ta_mod_input as ta_input
stderr Introspecting scheme=mscs_storage_table:    File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/splunktamscs/splunktaucclib/data_collection/ta_mod_input.py", line 15, in module
stderr Introspecting scheme=mscs_storage_table:      from splunktaucclib.data_collection import ta_checkpoint_manager as cpmgr
stderr Introspecting scheme=mscs_storage_table:  ImportError: cannot import name ta_checkpoint_manager
Introspecting scheme=mscs_storage_table: script running failed (exited with code 1).
Unable to initialize modular input "mscs_storage_table" defined in the app "Splunk_TA_microsoft-cloudservices": Introspecting scheme=mscs_storage_table: script running failed (exited with code 1)..

Unfortunately I have very little experience with modular inputs and I had never been facing this kind of problem.
Could someone please help me? It would be higly appreciated. Thank you very much in advance.
Best regards
Lukas Mecir

Labels (1)
1 Solution

tomasmoser
Contributor

Hi,

This issue was fixed. We found out that we had broken add-on Splunk TA for MS Cloud Services. Lot's of files that were supposed to be present in a Gitlab repository for this add-on were missing. Uploading and installing new add-on fixed all issues.

Tomas

View solution in original post

0 Karma

sean_wong
Explorer

Hi,

Can you described more about "uploading and installing new add-on"?
I tried to uploaded and re-install Splunk_TA_microsoft-cloudservices but the problem is still occurs. Can you advise how to fix it? Thanks.

KevinMurray
Explorer

Chances are that some python scripts have "invalid syntax" errors.

Check your splunkd logs for "mscs_storage" ERROR events.

You may find the scripts:

/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_blob.py

/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/mscs_storage_table.py

have this error:

print(hlp % sys.argv[0], file=sys.stderr) <-- the equal sign is INVALID!!!!!

During my conversion to Splunk 8, a lot of these problem scripts are appearing although they are supposed to be Splunk 8 compatible. 

 

 

 

 

0 Karma

KevinMurray
Explorer

Splunkd logs may have this error:

raise ImportError('Unable to load system certificate authority files')

ImportError: Unable to load system certificate authority files

looks like it is being generated by this script:

/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/bin/splunktamscs/ca_certs_locater.py

plaftaric
Explorer

Switching python from 3 to 2 (as recommended by bpaul_splunk) did not work for me (Using latest Splunk 8.0.5).

I still get the same error as originally reported by lukasmecir, and now by KevinMurray.

0 Karma

vanvan
Path Finder

Splunk 8.x and Python 2.x is supposed to be a "no go"

0 Karma

Armando
Explorer

Does your Splunk box where the input lives have Python 2 installed? I had to manually install it on my CentOS 8 box as the default for that distro is Python 3. Also, did you make any other changes besides what was outlined? If you did, I'd recommend removing and reinstalling the MSCS app and rebooting Splunk after making the Python changes. That's what ended up working for me.

0 Karma

plaftaric
Explorer

Does your Splunk box where the input lives have Python 2 installed?

Yep.

I have CentOS Linux release 7.7, and Python 2 is already here

Also, did you make any other changes besides what was outlined?

Nope.

I  tried again, clean install no luck. Downgrade to python 2 no luck.

 

Thanks anyway.

 

0 Karma

KevinMurray
Explorer

I tried using both versions of python.  That is how I produced the top 2 entries in this communication stream. 

My back end for Splunk is SUSE 15 SP1. 

I wonder if there are components of python, other libraries, etc.  that are needed to support Splunk apps and add-ons that are not present in every Linux distro that the developer may have in their environment?

It would be nice if the dependencies were listed in the app/add-on entries in splunkbase to assist in troubleshooting.

Thanks to all that have replied to this stream so far, I hope we find a solution soon!

 

 

 

0 Karma

tomasmoser
Contributor

Hi,

This issue was fixed. We found out that we had broken add-on Splunk TA for MS Cloud Services. Lot's of files that were supposed to be present in a Gitlab repository for this add-on were missing. Uploading and installing new add-on fixed all issues.

Tomas

0 Karma

Armando
Explorer

Would you be able to be more specific with your solution (version, source of app package, etc.)? I'm having the same issue described by OP as of 8/4/2020. I've tried installing from Splunk Web and by manually copying the app folder to the appropriate Splunk directory. Neither one of those worked.

0 Karma

bpaul_splunk
Splunk Employee
Splunk Employee

I suspect that may be an issue with Python compatibility.  Try forcing the App to use Python 2 by doing the following.

  1. Make a new directory called “local” in $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices.
  2. Copy the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/default/inputs.conf and $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/default/restmap.conf files to the new $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory.
  3. Edit the inputs.conf and restmap.conf files in the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory and change “python3” to “python2”.
  4. Restart Splunk.

If that works, you can remove the changes in the future by deleting the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft_cloudservices/local directory when the compatibility issue is corrected.

0 Karma

louismai
Path Finder

This error is raised in version 4.1.3 too.

I have changed the inputs.conf and restmap.conf to python2, but it doesn't work

Splunk: 7.7.3

Python: 2.7

0 Karma

Dave1
Observer

This worked for me as well.  Thanks for posting this! This is really frustrating trying to figure it out.  Knowing this workaround is frustrating because I'm using the unmodified Splunk docker image.  

0 Karma

Armando
Explorer

Yeah. I had to manually install Python2 because CentOS 8 doesn't appear to have it by default. Other than that, config seems to be working as intended. I was able to bring in storage table data containing Windows Event logs from VMs. I'm currently working on formatting that data to fit the syntax that the Windows TA uses to parse fields out, etc. 

0 Karma

Armando
Explorer

I tried what you said and so far so good! I haven't really had a chance to create the necessary configuration on the Azure side, but the error within Splunk cleared. I'll report back tomorrow once I've had a chance to do the Azure end of the deal. Thanks!

0 Karma

plaftaric
Explorer

What version is the "new" add-on?  Currently latest 4.0.1 reports the same problem.

amyers16
Path Finder

I am in the same boat with this one. Anyone have suggestions?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...