Splunk Dev

How to pass credentials within custom search command using Python SDK 1.5

elekanne
Explorer

I'm building a custom search command (in Python with the SDK 1.5). From within the script (search command), I need to connect to the Splunk instance to execute some extra searches and retrieve some kv collections.

To have a service object to Splunk I use:

service = client.connect( username=, password=, app=)

But I do not want to have the username and password part of this script. I have tried "passauth" from the commands.conf, but I do not seem to get a token or session.

Is there a way to pass credentials to this type of script?

0 Karma
1 Solution

elekanne
Explorer

I found the answer. If i do:

logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)
for app in service.apps:
    logger.debug("apps: %s", app['name'])

Then the session token is retrieved. The generating command derives from a search command which has this information.
The same applies for getting the version of splunk you're running which is self._metadata.searchinfo.splunk_version

View solution in original post

curben
Observer

You can directly use the service object which is already authenticated.

 

service = self.service
logger.debug(f"Splunk version: {service.info.version}")

 

Note that the namespace is app-specific and any item created will be owned by the user that runs the command.

For example, I run the following search command as user1 in helloworld app.

 

service = self.service
storage_passwords = service.storage_passwords
storage_password = storage_passwords.create("password1", "bobuser")
logger.debug(f"Successfully saved password for {storage_password.name} user")

 

 This is equivalent to:

 

service = client.connect(
  token=self._metadata.searchinfo.session_key,
  owner="user1",
  app="helloworld",
  sharing="app"
)
storage_passwords = service.storage_passwords
...

 

 TL;DR if the operation is within an app, just use the "service" object; for cross-app operation, use "self._metadata.searchinfo.session_key" and specify "app" parameter.

0 Karma

elekanne
Explorer

I found the answer. If i do:

logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)
for app in service.apps:
    logger.debug("apps: %s", app['name'])

Then the session token is retrieved. The generating command derives from a search command which has this information.
The same applies for getting the version of splunk you're running which is self._metadata.searchinfo.splunk_version

insightfinder
Engager

This worked for me. Thanks.

0 Karma

Yasaswy
Contributor

Hi,
session key is passed along when script gets triggered. You can capture it by doing something like:

tokn = sys.stdin.readline().strip()

token will now hold the session key. It is urlencoded so will need to decode it. Typically ... if you are on the recent versions of splunk, something like below should work for you to connect:

tokn = tokn.replace("sessionKey=", "")
tokn = urllib2.unquote(tokn).decode('utf8')
service = client.connect(token = tokn, app = yourapp)

0 Karma

elekanne
Explorer

Thanks for your code. By looking at it I do see the difference. You are not using the Splunk Python SDK where i do use that.
But by searching through the Splunk Python SDK code myself i found what i was looking for.

I told you that i need the service object to connect to Splunk and what i found out is that this Splunk Python SDK already provides me that.
When I use the following with the generate(self) function i do not even have to know the token i immediately can use the service object.

for app in self.service.apps:
      logger.debug("apps: %s", app['name'])

The Splunk Python SDK delivers a client which wraps around a couple of the REST services. If i need to call the not already covered REST services i still need the token which i found i can easily retrieve by doing:

logger.debug("session token key is: %s", self._metadata.searchinfo.session_key)
service = client.connect( token=self._metadata.searchinfo.session_key)

This is just logging the token and use it for recreate the service object myself.

So thanks for thinking with me it brought me to look deeper within the Splunk Python SDK to find these answers. I have just written them here because this is definitely someone else is looking for either.

0 Karma

elekanne
Explorer

Tx again for the comment. I'm running Splunk 6.3.2 with python SDK 1.5.0.2 just to be sure. I tried to get the information from the sys.stdin but nothing is there, it is not containing something like a session or token. So the variable "tokn" stays empty. And i do fire of the search command from out of splunk with logged in as admin.
But indeed this is how i have understood how it should work.
Once i do have the session or token i indeed can connect.

0 Karma

Yasaswy
Contributor

ahh.. I did quick test and looks like for search commands stdin only holds the location of csv file with the requested headers. I was able to get the token by reading the csv file..

csvf = sys.stdin.readline().strip()
csvf = csvf.replace("infoPath:", "")
with open(csvf) as hfile:
        reader = csv.DictReader(hfile)
        for row in reader:
                tkn = row['_auth_token']

As mentioned above if the sessionkey might be urlencoded in which case you will need to decode before passing it to client connect. Hope this works out ..

0 Karma

elekanne
Explorer

Ok this all doesn't seems to work at my environment. I have to say I'm also running El Capitan 10.11.3.

Commands.conf:

[ucmstest]
filename = ucmstest.py
supports_getinfo = true
supports_rawargs = false
passauth = true
enableheader = true
requires_srinfo = true

And ucmstest.py is:

from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators 
import sys, os
import time
import logging, logging.handlers
import splunk
import urllib2


###passauth=True; enableheader=True
@Configuration()
class ucmstest(GeneratingCommand):

    count = Option(require=True, validate=validators.Integer(0))
    text = Option(require=True)

    def generate(self):
        logger = logging.getLogger('UCMS dq Logging')  
        SPLUNK_HOME = os.environ['SPLUNK_HOME']
        LOGGING_DEFAULT_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log.cfg')
        LOGGING_LOCAL_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log-local.cfg')
        LOGGING_STANZA_NAME = 'pythondebug'
        LOGGING_FILE_NAME = "ucmsdqreconsilestats.log"
        BASE_LOG_PATH = os.path.join('var', 'log', 'splunk')
        LOGGING_FORMAT = "%(asctime)s %(levelname)-s\t%(module)s:%(lineno)d - %(message)s"
        splunk_log_handler = logging.handlers.RotatingFileHandler(os.path.join(SPLUNK_HOME, BASE_LOG_PATH, LOGGING_FILE_NAME), mode='a') 
        splunk_log_handler.setFormatter(logging.Formatter(LOGGING_FORMAT))
        logger.addHandler(splunk_log_handler)
        logmode = logging.DEBUG
        logger.setLevel(logmode)
        splunk.setupSplunkLogger(logger, LOGGING_DEFAULT_CONFIG_FILE, LOGGING_LOCAL_CONFIG_FILE, LOGGING_STANZA_NAME)

        logger.info("Start of logging...................................................")
        yield {'_serial': 1, '_time': time.time(), '_raw':"this is a test"}
        logger.debug("sys.stdin is: %s",str(sys.stdin))

        #logger.debug("read %s", str(sys.stdin.read()))
        csvf = sys.stdin.readline().strip()
        logger.debug("csvf file is: %s and length %i", csvf, len(csvf))
        #csvf = csvf.replace("infoPath:", "")
        logger.debug("csvf is now: %s", csvf)
        with open(csvf) as hfile:
            logger.debug("within open section")
            reader = csv.DictReader(hfile)
            for row in reader:
                tkn = row['_auth_token']
                logger.debug("tkn is: %s", tkn)

dispatch(ucmstest, sys.argv, sys.stdin, sys.stdout, __name__)

When i do |ucmstest count=1 text="bla" in Splunk Web, I do get:

External search command 'ucmstest' returned error code 1. Script output = "error_message=IOError at "/Applications/splunkucmsdq/splunk/etc/apps/ucmsdq/bin/ucmstest.py", line 41 : [Errno 2] No such file or directory: '' _raw,__mv__raw,_time,__mv__time,_serial,__mv__serial this is a test,,1456854995.48,,1, "

And i do get the following in my "ucmsdqreconsilestats.log":

2016-03-01 19:03:43,565 INFO    __init__:168 - Using default logging config file: /Applications/splunkucmsdq/splunk/etc/log.cfg
2016-03-01 19:03:43,566 INFO    __init__:206 - Setting logger=splunk level=INFO
2016-03-01 19:03:43,566 INFO    __init__:206 - Setting logger=splunk.appserver level=INFO
2016-03-01 19:03:43,567 INFO    __init__:206 - Setting logger=splunk.appserver.controllers level=INFO
2016-03-01 19:03:43,567 INFO    __init__:206 - Setting logger=splunk.appserver.controllers.proxy level=INFO
2016-03-01 19:03:43,567 INFO    __init__:206 - Setting logger=splunk.appserver.lib level=WARN
2016-03-01 19:03:43,567 INFO    __init__:206 - Setting logger=splunk.pdfgen level=INFO
2016-03-01 19:03:43,567 INFO    __init__:206 - Setting logger=splunk.search level=INFO
2016-03-01 19:03:43,567 INFO    ucmstest:32 - Start of logging...................................................
2016-03-01 19:03:43,567 DEBUG   ucmstest:34 - sys.stdin is: <open file '<stdin>', mode 'r' at 0x107dc10c0>
2016-03-01 19:03:43,568 DEBUG   ucmstest:38 - csvf file is:  and length 0
2016-03-01 19:03:43,568 DEBUG   ucmstest:40 - csvf is now:

Can you otherwise share your code?

0 Karma

elekanne
Explorer

I also have to say that i did not installed the SDK but took the "splunklib" folder out of the SDK to put it in ../bin/. folder of my app.

0 Karma

Yasaswy
Contributor

Sure.. I have the following in my commands.conf (located in SPLUNK_HOME/etc/apps/MYAPP/local)

[tkntest]
type = python
filename = tkntst.py
supports_getinfo = false
supports_rawargs = true
passauth = true
enableheader = true
outputheader = true
requires_srinfo = true
support_multivalues = true

tkntst.py (located in SPLUNK_HOME/etc/apps/MYAPP/bin)

import sys, os, logging, csv
from logging.handlers import TimedRotatingFileHandler

splkhome = os.environ['SPLUNK_HOME']
splvar = os.path.join(splkhome, 'var', 'log', 'splunk', 'sctest.log')

### Set Logging
log = logging.getLogger('tkntst')
log.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s')
handler = logging.handlers.TimedRotatingFileHandler(splvar,when="d",interval=1,backupCount=1)
handler.setFormatter(formatter)
log.addHandler(handler)

try:
        log.info('Search commnd initiated')
        csvf = sys.stdin.readline().strip()
        csvf = csvf.replace("infoPath:", "")
        log.info('Successfully received csvf from Splunk %s' % csvf)

        with open(csvf) as hfile:
                reader = csv.DictReader(hfile)
                for row in reader:
                        tkn = row['_auth_token']
        log.info('Successfully received token from Splunk %s' % tkn)

except Exception, e:
        log.error('Unable to get session %s' % str(e))
        sys.exit()

Ran in search (MYAPP) "|tkntest"

tail /xxx/xxx/splunk/var/log/splunk/sctest.log
2016-03-01 16:58:31,932 [INFO] Search commnd initiated
2016-03-01 16:58:31,932 [INFO] Successfully received csvf from  /xxx/xxx/splunk/var/run/splunk/dispatch/1456869511.102/eternSearchResultsInfo.csv
2016-03-01 16:58:31,933 [INFO] Successfully received token from Splunk Ps^29NN_8U3WSwXdlt897cDV8bsOwxsXfN9Tqv9ATql77OONJS6S6gGI9lq3vNSW5Szwc6ltPNjFhg9HADwwpGS72CMJFTqcktxgS^8t9mrbtYoKE1T7KN

Also since you have the splunklib... checkout this answer ... seems much simpler

0 Karma

Yasaswy
Contributor

Hi, If the script is being called from splunk the best way to do this is with tokens. Can you explain how you tried to use passauth? Provide some code?

0 Karma

elekanne
Explorer

He tx for reply.

My code is the following:
commands.conf:
[ucmstest]
filename = ucmstest.py
supports_getinfo = true
supports_rawargs = true
passauth = true
enableheader = true
outputheader = true
requires_srinfo = true
support_multivalues = true

ucmstest.py:

from splunklib.searchcommands import dispatch, GeneratingCommand, Configuration, Option, validators 
import sys, os
import time
import splunklib.client as client
import splunklib.results as results
import splunklib.binding as binding
import logging, logging.handlers
import splunk

###passauth=True; enableheader=True
@Configuration()
class ucmstest(GeneratingCommand):

    count = Option(require=True, validate=validators.Integer(0))
    text = Option(require=True)

    def generate(self):
        logger = logging.getLogger('UCMS dq Logging')  
        SPLUNK_HOME = os.environ['SPLUNK_HOME']
        LOGGING_DEFAULT_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log.cfg')
        LOGGING_LOCAL_CONFIG_FILE = os.path.join(SPLUNK_HOME, 'etc', 'log-local.cfg')
        LOGGING_STANZA_NAME = 'pythondebug'
        LOGGING_FILE_NAME = "ucmsdqreconsilestats.log"
        BASE_LOG_PATH = os.path.join('var', 'log', 'splunk')
        LOGGING_FORMAT = "%(asctime)s %(levelname)-s\t%(module)s:%(lineno)d - %(message)s"
        splunk_log_handler = logging.handlers.RotatingFileHandler(os.path.join(SPLUNK_HOME, BASE_LOG_PATH, LOGGING_FILE_NAME), mode='a') 
        splunk_log_handler.setFormatter(logging.Formatter(LOGGING_FORMAT))
        logger.addHandler(splunk_log_handler)
        logmode = logging.DEBUG
        logger.setLevel(logmode)
        splunk.setupSplunkLogger(logger, LOGGING_DEFAULT_CONFIG_FILE, LOGGING_LOCAL_CONFIG_FILE, LOGGING_STANZA_NAME)
        yield {'_serial': 1, '_time': time.time(), '_raw':"this is a test"}

        logger.info("Start of logging...................................................")
        for line in sys.argv:
            logger.debug("value of argv is: %s", line)

dispatch(ucmstest, sys.argv, sys.stdin, sys.stdout, __name__)

When called from the splunk search line: |ucmstest count=5 text="hallo" it is indeed outputting what i have put in yield.
In the "ucmsdqreconsilestats.log" i can then see the values of sys.argv but no token or session is there. Also nothin in sys.stdin.
And from out this command i want to do some extra searches, retrieving a CIM model and the data of a CIM model. So i need to connect to the service by:
service = client.connect(something)
And that something is either "username=, password=" or something with token or session.

But how to get in the token or session value?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...