Getting Data In

How do you test a modular input on the command line with a valid session_key?

envancleve
Engager

I'm writing a modular input (simpleinput) which stores its password using StoragePasswords facility offered by the Splunk Python SDK.

The script works just fine when its called by Splunk itself. However, when I try to run the module on the command line for testing purposes, the call always fails saying the "Session is not logged in".

Here's how I run the script on the command line:

$SPLUNK_HOME\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc | "C:\Program Files\Splunk\bin\splunk" cmd python "c:\Program Files\Splunk\etc\apps\simpleinput\bin\simpleinput.py"

It appears that the SessionKey produced by the print-modinput-config command is invalid. Below, I've shown a session which attempts to use the SessionKey generated by the print-modinput-config to authenticate to the /services/server/info endpoint. After it fails, I grab a key manually by authenticating with a username and password, which is successful.

How do other developers run their modular inputs while testing? Is there a way to get a valid sessionkey out of the print-modinput-config command?

$SPLUNK_HOME\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc

<?xml version="1.0" encoding="UTF-8"?>
<input>
  <server_host>215CN72</server_host>
  <server_uri>https://127.0.0.1:8089</server_uri>
  <session_key>session_key_from_print_modinput_config</session_key>
  <checkpoint_dir>C:\Program Files\Splunk\var\lib\splunk\modinputs\simpleinput</checkpoint_dir>
  <configuration>
    <stanza name="simpleinput://evc" app="launcher">
      <param name="disabled">0</param>
      <param name="first_name">Eric</param>
      <param name="host">215CN72</param>
      <param name="index">default</param>
      <param name="interval">60</param>
    </stanza>
  </configuration>
</input>

curl -k "https://localhost:8089/services/server/info" -H "Authorization: Splunk session_key_from_print_modinput_config"

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="WARN">call not properly authenticated</msg>
  </messages>
</response>

curl -k "https://localhost:8089/services/auth/login" -d "username=admin&password=changeme"

<response>
  <sessionKey>session_key_from_admin_login</sessionKey>
</response>

curl -k "https://localhost:8089/services/server/info" -H "Authorization: Splunk session_key_from_admin_login"

<?xml version="1.0" encoding="UTF-8"?>
<!--This is to override browser formatting; see server.conf[httpServer] to disable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-->
<?xml-stylesheet type="text/xml" href="/static/atom.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:s="http://dev.splunk.com/ns/rest" xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">
  <title>server-info</title>
  <id>https://localhost:8089/services/server/info</id>
snip snip snip snip
1 Solution

envancleve
Engager

In case this helps someone in the future, I worked around the problem by cheating. I wrote a small script to manually authenticate and generate a session key. Then I replace the session key in the modinput config and pass it to the script in question.

So, to run it on the command line, I type:

"C:\Program Files\Splunk\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc | "c:\Program Files (x86)\Python36-32\python" C:\Utils\session_replacer.py | "C:\Program Files\Splunk\bin\splunk" cmd python "c:\Program Files\Splunk\etc\apps\simpleinput\bin\simpleinput.py"

Here's the content of the file session_replacer.py:

#!/usr/bin/env python

import sys
import re
import requests
import urllib3
urllib3.disable_warnings()

splunk_url = 'https://localhost:8089/services/auth/login'
username = 'admin'
password = 'changeme'

urllib3.disable_warnings()
session = requests.Session()
session.verify = False

form = {'username': username, 'password': password}
response = session.post(url=splunk_url, data=form, verify=False)
matches = re.search('<sessionKey>(.+)</sessionKey>', response.text, re.MULTILINE|re.DOTALL)
if matches is not None:
    new_key = matches.group(1)

input = sys.stdin.read()
output = re.sub('<session_key>(.+)</session_key>','<session_key>{}</session_key>'.format(new_key), input, re.MULTILINE|re.DOTALL)
print(output)

View solution in original post

yoho
Contributor

I found another workaround.

Just launch command $SPLUNK_HOME/bin/splunk login and provide requested credentials (ideally admin level). It will create a session key within a file named $HOME/.splunk/auth*. Within this file, you will find a valid session key which should work for your modular input (until the session timeout value, I think it's 2h by default).

Caution that the session key is between <sessionkey> tags and your modular input expects <session_key> instead

0 Karma

envancleve
Engager

In case this helps someone in the future, I worked around the problem by cheating. I wrote a small script to manually authenticate and generate a session key. Then I replace the session key in the modinput config and pass it to the script in question.

So, to run it on the command line, I type:

"C:\Program Files\Splunk\bin\splunk" cmd splunkd print-modinput-config simpleinput simpleinput://evc | "c:\Program Files (x86)\Python36-32\python" C:\Utils\session_replacer.py | "C:\Program Files\Splunk\bin\splunk" cmd python "c:\Program Files\Splunk\etc\apps\simpleinput\bin\simpleinput.py"

Here's the content of the file session_replacer.py:

#!/usr/bin/env python

import sys
import re
import requests
import urllib3
urllib3.disable_warnings()

splunk_url = 'https://localhost:8089/services/auth/login'
username = 'admin'
password = 'changeme'

urllib3.disable_warnings()
session = requests.Session()
session.verify = False

form = {'username': username, 'password': password}
response = session.post(url=splunk_url, data=form, verify=False)
matches = re.search('<sessionKey>(.+)</sessionKey>', response.text, re.MULTILINE|re.DOTALL)
if matches is not None:
    new_key = matches.group(1)

input = sys.stdin.read()
output = re.sub('<session_key>(.+)</session_key>','<session_key>{}</session_key>'.format(new_key), input, re.MULTILINE|re.DOTALL)
print(output)
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...