topic Re: Splunk Stream: Failed to detect Splunk_TA_stream status in All Apps and Add-ons

Splunk Stream: Failed to detect Splunk_TA_stream status

atsichlis — Tue, 29 Sep 2020 16:40:09 GMT

I just installed the Stream App on an on-prem heavy fowarder and when I select the "Collect data from this machine using Wire Data input (Splunk_TA_stream)." I get the following error:

Failed to detect Splunk_TA_stream status.

the splunk_app_stream log shows me:

Error getting the streamfwd auth, return streamfwd auth is disabled

Has anyone encountered this issue? If so can you please provide insight on how to solve it?

Regards,

Re: Splunk Stream: Failed to detect Splunk_TA_stream status

vshcherbakov_sp — Tue, 29 Sep 2020 16:39:33 GMT

Seems like there's something wrong with the Stream app install.. There should be exception info logged before the error you're quoting. Can you provide a larger snippet of splunk_app_stream.log file around the error?

Re: Splunk Stream: Failed to detect Splunk_TA_stream status

atsichlis — Tue, 29 Sep 2020 16:40:35 GMT

Thank you for getting back!

More error details below:

2017-11-08 15:14:49,980 ERROR streams_utils:270 - Error getting the streamfwd auth, return streamfwd auth is disabled
2017-11-08 15:14:54,559 ERROR stream_kvstore_utils:115 - KV store failed to start, setting the kv store fatal error flag to true
2017-11-08 15:14:54,559 INFO stream_kvstore_utils:177 - is_kv_store_ready, kv store status :: failed
2017-11-08 15:14:54,559 INFO stream_kvstore_utils:178 - search_head_shc_member:: server_roles [u'license_master', u'deployment_server']
2017-11-08 15:14:54,559 ERROR stream_kvstore_utils:200 - kv_store_rest_request: Timedout waiting for KVstore status False to be ready
2017-11-08 15:14:54,559 ERROR stream_kvstore_utils:340 - read_kv_store_apps_meta: Error getting apps meta from kv store collection, reason Timedout waiting for KVstore status to be ready
2017-11-08 15:14:54,559 ERROR stream_kvstore_utils:193 - kv_store_rest_request: fatal error kv store failed to start
2017-11-08 15:14:54,559 ERROR streamfwdauth:62 - expected string or buffer
Traceback (most recent call last):
File "E:\Program Files\Splunk\etc\apps\splunk_app_stream\bin\splunk_app_stream\models\streamfwdauth.py", line 53, in get
return read_from_kv_store_coll(streamfwd_auth_kv_store_with_session_key_uri, sessionKey)
File "E:\Program Files\Splunk\etc\apps\splunk_app_stream\bin\stream_kvstore_utils.py", line 277, in read_from_kv_store_coll
jsonResp = json.loads(serverContent)
File "E:\Program Files\Splunk\Python-2.7\Lib\json_init.py", line 339, in loads
return _default_decoder.decode(s)
File "E:\Program Files\Splunk\Python-2.7\Lib\json\decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer
2017-11-08 15:14:54,980 ERROR streams_utils:269 - [HTTP 500] Splunkd internal error; []
Traceback (most recent call last):
File "C:\Program Files\Splunk\etc\apps\splunk_app_stream\bin\stream_utils.py", line 262, in validate_streamfwd_auth
timeout=15
File "C:\Program Files\Splunk\Python-2.7\Lib\site-packages\splunk\rest__init_.py", line 564, in simpleRequest
raise splunk.InternalServerError, (None, serverResponse.messages)
InternalServerError: [HTTP 500] Splunkd internal error; []
2017-11-08 15:14:54,980 ERROR streams_utils:270 - Error getting the streamfwd auth, return streamfwd auth is disabled

Re: Splunk Stream: Failed to detect Splunk_TA_stream status

vshcherbakov_sp — Wed, 08 Nov 2017 21:00:43 GMT

Thanks for providing the log; I believe the problem is due to the KV store being not operational. I'd suggest looking at the mongod.log to see if it's due to expired SSL certificate (the most likely cause per my experience) or some other issue..

Re: Splunk Stream: Failed to detect Splunk_TA_stream status

gordo32 — Thu, 30 Sep 2021 22:09:45 GMT

I resolved this in part due to Raúl Marín's excellent writeup and youtube video (https://raulmarin.me/2020/04/26/netflow-traffic-ingestion-with-splunk-stream-and-an-independent-stream-forwarder/ & https://www.youtube.com/watch?v=Usjy5NF0rwE, respectively).

He was dealing with a similar issue of errors when choosing the 2nd option (Collect data from other machines). It turns out *both* options will give an error if the host's name isn't defined in /etc/system/local/inputs.conf.

So, by adding this stanza, and restarting Splunk, that problem went away:

[default] host = splunk-hostname

The wizard then started to prompt me to run the set_permissions.sh command. After adjusting permissions on the script using the command below, then running the script and restarting Splunk again, everything went smoothly

sudo chmod +x ./set_permissions.sh

Thanks,

Gord T.