I'm building a custom search command (in Python with the SDK 1.5). From within the script (search command), I need to connect to the Splunk instance to execute some extra searches and retrieve some kv collections.
To have a service object to Splunk I use:
service = client.connect( username=, password=, app=)
But I do not want to have the username and password part of this script. I have tried "passauth" from the commands.conf, but I do not seem to get a token or session.
Is there a way to pass credentials to this type of script?
I found the answer. If i do:
logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)
for app in service.apps:
logger.debug("apps: %s", app['name'])
Then the session token is retrieved. The generating command derives from a search command which has this information.
The same applies for getting the version of splunk you're running which is self._metadata.searchinfo.splunk_version
You can directly use the service object which is already authenticated.
service = self.service
logger.debug(f"Splunk version: {service.info.version}")
Note that the namespace is app-specific and any item created will be owned by the user that runs the command.
For example, I run the following search command as user1 in helloworld app.
service = self.service
storage_passwords = service.storage_passwords
storage_password = storage_passwords.create("password1", "bobuser")
logger.debug(f"Successfully saved password for {storage_password.name} user")
This is equivalent to:
service = client.connect(
token=self._metadata.searchinfo.session_key,
owner="user1",
app="helloworld",
sharing="app"
)
storage_passwords = service.storage_passwords
...
TL;DR if the operation is within an app, just use the "service" object; for cross-app operation, use "self._metadata.searchinfo.session_key" and specify "app" parameter.
I found the answer. If i do:
logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)
for app in service.apps:
logger.debug("apps: %s", app['name'])
Then the session token is retrieved. The generating command derives from a search command which has this information.
The same applies for getting the version of splunk you're running which is self._metadata.searchinfo.splunk_version
This worked for me. Thanks.
Hi,
session key is passed along when script gets triggered. You can capture it by doing something like:
tokn = sys.stdin.readline().strip()
token will now hold the session key. It is urlencoded so will need to decode it. Typically ... if you are on the recent versions of splunk, something like below should work for you to connect:
tokn = tokn.replace("sessionKey=", "")
tokn = urllib2.unquote(tokn).decode('utf8')
service = client.connect(token = tokn, app = yourapp)
Thanks for your code. By looking at it I do see the difference. You are not using the Splunk Python SDK where i do use that.
But by searching through the Splunk Python SDK code myself i found what i was looking for.
I told you that i need the service object to connect to Splunk and what i found out is that this Splunk Python SDK already provides me that.
When I use the following with the generate(self) function i do not even have to know the token i immediately can use the service object.
for app in self.service.apps:
logger.debug("apps: %s", app['name'])
The Splunk Python SDK delivers a client which wraps around a couple of the REST services. If i need to call the not already covered REST services i still need the token which i found i can easily retrieve by doing:
logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)
This is just logging the token and use it for recreate the service object myself.
So thanks for thinking with me it brought me to look deeper within the Splunk Python SDK to find these answers. I have just written them here because this is definitely someone else is looking for either.
Tx again for the comment. I'm running Splunk 6.3.2 with python SDK 1.5.0.2 just to be sure. I tried to get the information from the sys.stdin but nothing is there, it is not containing something like a session or token. So the variable "tokn" stays empty. And i do fire of the search command from out of splunk with logged in as admin.
But indeed this is how i have understood how it should work.
Once i do have the session or token i indeed can connect.
ahh.. I did quick test and looks like for search commands stdin only holds the location of csv file with the requested headers. I was able to get the token by reading the csv file..
csvf = sys.stdin.readline().strip()
csvf = csvf.replace("infoPath:", "")
with open(csvf) as hfile:
reader = csv.DictReader(hfile)
for row in reader:
tkn = row['_auth_token']
As mentioned above if the sessionkey might be urlencoded in which case you will need to decode before passing it to client connect. Hope this works out ..
Ok this all doesn't seems to work at my environment. I have to say I'm also running El Capitan 10.11.3.
Commands.conf:
[ucmstest]
filename = ucmstest.py
supports_getinfo = true
supports_rawargs = false
passauth = true
enableheader = true
requires_srinfo = true
And ucmstest.py is:
from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators
import sys, os
import time
import logging, logging.handlers
import splunk
import urllib2
###passauth=True; enableheader=True
@Configuration()
class ucmstest(GeneratingCommand):
count = Option(require=True, validate=validators.Integer(0))
text = Option(require=True)
def generate(self):
logger = logging.getLogger('UCMS dq Logging')
SPLUNK_HOME = os.environ['SPLUNK_HOME']
LOGGING_DEFAULT_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log.cfg')
LOGGING_LOCAL_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log-local.cfg')
LOGGING_STANZA_NAME = 'pythondebug'
LOGGING_FILE_NAME = "ucmsdqreconsilestats.log"
BASE_LOG_PATH = os.path.join('var', 'log', 'splunk')
LOGGING_FORMAT = "%(asctime)s %(levelname)-s\t%(module)s:%(lineno)d - %(message)s"
splunk_log_handler = logging.handlers.RotatingFileHandler(os.path.join(SPLUNK_HOME, BASE_LOG_PATH, LOGGING_FILE_NAME), mode='a')
splunk_log_handler.setFormatter(logging.Formatter(LOGGING_FORMAT))
logger.addHandler(splunk_log_handler)
logmode = logging.DEBUG
logger.setLevel(logmode)
splunk.setupSplunkLogger(logger, LOGGING_DEFAULT_CONFIG_FILE, LOGGING_LOCAL_CONFIG_FILE, LOGGING_STANZA_NAME)
logger.info("Start of logging...................................................")
yield {'_serial': 1, '_time': time.time(), '_raw':"this is a test"}
logger.debug("sys.stdin is: %s",str(sys.stdin))
#logger.debug("read %s", str(sys.stdin.read()))
csvf = sys.stdin.readline().strip()
logger.debug("csvf file is: %s and length %i", csvf, len(csvf))
#csvf = csvf.replace("infoPath:", "")
logger.debug("csvf is now: %s", csvf)
with open(csvf) as hfile:
logger.debug("within open section")
reader = csv.DictReader(hfile)
for row in reader:
tkn = row['_auth_token']
logger.debug("tkn is: %s", tkn)
dispatch(ucmstest, sys.argv, sys.stdin, sys.stdout, __name__)
When i do |ucmstest count=1 text="bla"
in Splunk Web, I do get:
External search command 'ucmstest' returned error code 1. Script output = "error_message=IOError at "/Applications/splunkucmsdq/splunk/etc/apps/ucmsdq/bin/ucmstest.py", line 41 : [Errno 2] No such file or directory: '' _raw,__mv__raw,_time,__mv__time,_serial,__mv__serial this is a test,,1456854995.48,,1, "
And i do get the following in my "ucmsdqreconsilestats.log":
2016-03-01 19:03:43,565 INFO __init__:168 - Using default logging config file: /Applications/splunkucmsdq/splunk/etc/log.cfg
2016-03-01 19:03:43,566 INFO __init__:206 - Setting logger=splunk level=INFO
2016-03-01 19:03:43,566 INFO __init__:206 - Setting logger=splunk.appserver level=INFO
2016-03-01 19:03:43,567 INFO __init__:206 - Setting logger=splunk.appserver.controllers level=INFO
2016-03-01 19:03:43,567 INFO __init__:206 - Setting logger=splunk.appserver.controllers.proxy level=INFO
2016-03-01 19:03:43,567 INFO __init__:206 - Setting logger=splunk.appserver.lib level=WARN
2016-03-01 19:03:43,567 INFO __init__:206 - Setting logger=splunk.pdfgen level=INFO
2016-03-01 19:03:43,567 INFO __init__:206 - Setting logger=splunk.search level=INFO
2016-03-01 19:03:43,567 INFO ucmstest:32 - Start of logging...................................................
2016-03-01 19:03:43,567 DEBUG ucmstest:34 - sys.stdin is: <open file '<stdin>', mode 'r' at 0x107dc10c0>
2016-03-01 19:03:43,568 DEBUG ucmstest:38 - csvf file is: and length 0
2016-03-01 19:03:43,568 DEBUG ucmstest:40 - csvf is now:
Can you otherwise share your code?
I also have to say that i did not installed the SDK but took the "splunklib" folder out of the SDK to put it in ../bin/. folder of my app.
Sure.. I have the following in my commands.conf (located in SPLUNK_HOME/etc/apps/MYAPP/local)
[tkntest]
type = python
filename = tkntst.py
supports_getinfo = false
supports_rawargs = true
passauth = true
enableheader = true
outputheader = true
requires_srinfo = true
support_multivalues = true
tkntst.py (located in SPLUNK_HOME/etc/apps/MYAPP/bin)
import sys, os, logging, csv
from logging.handlers import TimedRotatingFileHandler
splkhome = os.environ['SPLUNK_HOME']
splvar = os.path.join(splkhome, 'var', 'log', 'splunk', 'sctest.log')
### Set Logging
log = logging.getLogger('tkntst')
log.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s')
handler = logging.handlers.TimedRotatingFileHandler(splvar,when="d",interval=1,backupCount=1)
handler.setFormatter(formatter)
log.addHandler(handler)
try:
log.info('Search commnd initiated')
csvf = sys.stdin.readline().strip()
csvf = csvf.replace("infoPath:", "")
log.info('Successfully received csvf from Splunk %s' % csvf)
with open(csvf) as hfile:
reader = csv.DictReader(hfile)
for row in reader:
tkn = row['_auth_token']
log.info('Successfully received token from Splunk %s' % tkn)
except Exception, e:
log.error('Unable to get session %s' % str(e))
sys.exit()
Ran in search (MYAPP) "|tkntest"
tail /xxx/xxx/splunk/var/log/splunk/sctest.log
2016-03-01 16:58:31,932 [INFO] Search commnd initiated
2016-03-01 16:58:31,932 [INFO] Successfully received csvf from /xxx/xxx/splunk/var/run/splunk/dispatch/1456869511.102/eternSearchResultsInfo.csv
2016-03-01 16:58:31,933 [INFO] Successfully received token from Splunk Ps^29NN_8U3WSwXdlt897cDV8bsOwxsXfN9Tqv9ATql77OONJS6S6gGI9lq3vNSW5Szwc6ltPNjFhg9HADwwpGS72CMJFTqcktxgS^8t9mrbtYoKE1T7KN
Also since you have the splunklib... checkout this answer ... seems much simpler
Hi, If the script is being called from splunk the best way to do this is with tokens. Can you explain how you tried to use passauth? Provide some code?
He tx for reply.
My code is the following:
commands.conf:
[ucmstest]
filename = ucmstest.py
supports_getinfo = true
supports_rawargs = true
passauth = true
enableheader = true
outputheader = true
requires_srinfo = true
support_multivalues = true
ucmstest.py:
from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators
import sys, os
import time
import splunklib.client as client
import splunklib.results as results
import splunklib.binding as binding
import logging, logging.handlers
import splunk
###passauth=True; enableheader=True
@Configuration()
class ucmstest(GeneratingCommand):
count = Option(require=True, validate=validators.Integer(0))
text = Option(require=True)
def generate(self):
logger = logging.getLogger('UCMS dq Logging')
SPLUNK_HOME = os.environ['SPLUNK_HOME']
LOGGING_DEFAULT_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log.cfg')
LOGGING_LOCAL_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log-local.cfg')
LOGGING_STANZA_NAME = 'pythondebug'
LOGGING_FILE_NAME = "ucmsdqreconsilestats.log"
BASE_LOG_PATH = os.path.join('var', 'log', 'splunk')
LOGGING_FORMAT = "%(asctime)s %(levelname)-s\t%(module)s:%(lineno)d - %(message)s"
splunk_log_handler = logging.handlers.RotatingFileHandler(os.path.join(SPLUNK_HOME, BASE_LOG_PATH, LOGGING_FILE_NAME), mode='a')
splunk_log_handler.setFormatter(logging.Formatter(LOGGING_FORMAT))
logger.addHandler(splunk_log_handler)
logmode = logging.DEBUG
logger.setLevel(logmode)
splunk.setupSplunkLogger(logger, LOGGING_DEFAULT_CONFIG_FILE, LOGGING_LOCAL_CONFIG_FILE, LOGGING_STANZA_NAME)
yield {'_serial': 1, '_time': time.time(), '_raw':"this is a test"}
logger.info("Start of logging...................................................")
for line in sys.argv:
logger.debug("value of argv is: %s", line)
dispatch(ucmstest, sys.argv, sys.stdin, sys.stdout, __name__)
When called from the splunk search line: |ucmstest count=5 text="hallo" it is indeed outputting what i have put in yield.
In the "ucmsdqreconsilestats.log" i can then see the values of sys.argv but no token or session is there. Also nothin in sys.stdin.
And from out this command i want to do some extra searches, retrieving a CIM model and the data of a CIM model. So i need to connect to the service by:
service = client.connect(something)
And that something is either "username=, password=" or something with token or session.
But how to get in the token or session value?