How do you test a modular input on the command line with a valid session_key?

[envancleve]

I'm writing a modular input (simpleinput) which stores its password using StoragePasswords facility offered by the Splunk Python SDK.

The script works just fine when its called by Splunk itself. However, when I try to run the module on the command line for testing purposes, the call always fails saying the "Session is not logged in".

Here's how I run the script on the command line:

$SPLUNK_HOME\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc | "C:\Program Files\Splunk\bin\splunk" cmd python "c:\Program Files\Splunk\etc\apps\simpleinput\bin\simpleinput.py"

It appears that the SessionKey produced by the print-modinput-config command is invalid. Below, I've shown a session which attempts to use the SessionKey generated by the print-modinput-config to authenticate to the /services/server/info endpoint. After it fails, I grab a key manually by authenticating with a username and password, which is successful.

How do other developers run their modular inputs while testing? Is there a way to get a valid sessionkey out of the print-modinput-config command?

$SPLUNK_HOME\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc

<?xml version="1.0" encoding="UTF-8"?>
<input>
  <server_host>215CN72</server_host>
  <server_uri>https://127.0.0.1:8089</server_uri>
  <session_key>session_key_from_print_modinput_config</session_key>
  <checkpoint_dir>C:\Program Files\Splunk\var\lib\splunk\modinputs\simpleinput</checkpoint_dir>
  <configuration>
    <stanza name="simpleinput://evc" app="launcher">
      <param name="disabled">0</param>
      <param name="first_name">Eric</param>
      <param name="host">215CN72</param>
      <param name="index">default</param>
      <param name="interval">60</param>
    </stanza>
  </configuration>
</input>

curl -k "https://localhost:8089/services/server/info" -H "Authorization: Splunk session_key_from_print_modinput_config"

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="WARN">call not properly authenticated</msg>
  </messages>
</response>

curl -k "https://localhost:8089/services/auth/login" -d "username=admin&password=changeme"

<response>
  <sessionKey>session_key_from_admin_login</sessionKey>
</response>

curl -k "https://localhost:8089/services/server/info" -H "Authorization: Splunk session_key_from_admin_login"

<?xml version="1.0" encoding="UTF-8"?>
<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->
<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
  <title>server-info</title>
  <id>https://localhost:8089/services/server/info</id>
snip snip snip snip

[yoho]

I found another workaround.

Just launch command $SPLUNK_HOME/bin/splunk login and provide requested credentials (ideally admin level). It will create a session key within a file named $HOME/.splunk/auth*. Within this file, you will find a valid session key which should work for your modular input (until the session timeout value, I think it's 2h by default).

Caution that the session key is between <sessionkey> tags and your modular input expects <session_key> instead

[envancleve · Accepted solution]

In case this helps someone in the future, I worked around the problem by cheating. I wrote a small script to manually authenticate and generate a session key. Then I replace the session key in the modinput config and pass it to the script in question.

So, to run it on the command line, I type:

"C:\Program Files\Splunk\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc | "c:\Program Files (x86)\Python36-32\python" C:\Utils\session_replacer.py | "C:\Program Files\Splunk\bin\splunk" cmd python "c:\Program Files\Splunk\etc\apps\simpleinput\bin\simpleinput.py"

Here's the content of the file session_replacer.py:

#!/usr/bin/env python

import sys
import re
import requests
import urllib3
urllib3.disable_warnings()

splunk_url = 'https://localhost:8089/services/auth/login'
username = 'admin'
password = 'changeme'

urllib3.disable_warnings()
session = requests.Session()
session.verify = False

form = {'username': username, 'password': password}
response = session.post(url=splunk_url, data=form, verify=False)
matches = re.search('<sessionKey>(.+)</sessionKey>', response.text, re.MULTILINE|re.DOTALL)
if matches is not None:
    new_key = matches.group(1)

input = sys.stdin.read()
output = re.sub('<session_key>(.+)</session_key>','<session_key>{}</session_key>'.format(new_key), input, re.MULTILINE|re.DOTALL)
print(output)