I am trying to work through an issue and cannot seem to find a answer. I need to create a bash script that uses an auth token to make calls to the Splunk Rest API. The script will be used by a custom app that is located on a searchhead (linux). Token Authentication has been enabled, and I can verify that the Token is enabled and valid. I can use the Token to make ad-hoc Rest API requests via the terminal local to the searchhead. If I try to use it within the script I am seeing unexpected behavior.
The token seems to be able to make certain API calls in the script but it cannot retrieve the results of the call. An example would be I can use the token in the script to initiate a search. I will receive back the Sid of that search. If I use the same script to try to retrieve the search results using the Sid, I get back nothing. No error message, but just no output at all. If I manually re-run the same API request using the Token and the same Sid outside of the script on the terminal. It will return my search result as expected.
I have turned on debug logging for JsonWebTokenHandler and see no issue. This behavior happens if I use the bash interpreter to execute the script (./script.sh) or (/opt/splunk/etc/app/app_name/bin/script.sh). There is no difference if I execute the script with /opt/splunk/bin/splunk cmd <path_to_script>.
I remember reading that there is something strange about the local interpreter that could be causing this, but I cannot find it again on google. Does anybody know what might be causing this behavior or how I can overcome this issue?
Searchhead is running Splunk version 8.1.2 hosted on Ubuntu.
