dsdlsupport SplunkSearch not working in "Splunk App for data science and deep learning"

[ConsoleBotTryPC]

Hey guys,
Hope y'all are doing well!

I wanted to experiment with Splunk's Deep Learning module to perform some tasks.

As mentioned in the "barebone_template" there are two methods to pull data from splunk in.

Because I want the data to be live, I want to be able to run a search inside the Jupiter notebook itself, hence proceeding with method 1.

Method 1 is done using Splunk's "dsdlsupport" Python library. But when I used the same commands they have in their template, it throws the following error for their default settings:

I wanted to check if someone has faced/solved this issue already before diving into their source code myself.

Thank you and have a nice day 🙂

Best,

[Gabriel]

Hey there,

Your post is a couple months old, but since I stumbled into the same issue, I figured there will be more Splunkers in the future that encounter the same challenge and would appreciate if the solution is documented somewhere. The first part of my response lays out how to resolve the issue, in the second part I talk about why the issue arises in the first place.

Part 1 - How to resolve the issue

1. Apps > DSDL App > Configuration > Setup > Check the "Yes" box > Scroll to the "Splunk Access in Jupyter (optioinal)" section > Use the following settings:
2. Enable Splunk Access: Yes
3. Splunk Access Token: Paste your token here. If you dont have one, you can spawn a token under Settings > Tokens.
4. Splunk Host Address: Paste your host address here (in my case it has this format: 123.456.78.90)
5. Splunk Management Port: 8089 (This is the default, if you did not change it, you can use 8089)
6. Press "Test & Save"
7. Apps > DSDL App > Configuration > Containers > Start your container. If your container was already running, stop it and restart it
8. Apps > DSDL App > Configuration > Containers > Press the "JUPYTER LAB" button
9. Now open the "barebone_template.ipynb" in the /notebooks folder
10. Execute the code that pulls data from Splunk. Now it should work just fine.
    import libs.SplunkSearch as SplunkSearch
    search = SplunkSearch.SplunkSearch()

Part 2 - More details in case you are curious

Execute the following code in your jupyter notebook. Here you can inspect all os variables.

import os
os.environ

For us of interest are the following.

os.environ["splunk_access_host"]

os.environ["splunk_access_port"]

os.environ["splunk_access_token"]

If you haven't fixed the issue yet, os.environ["splunk_access_enabled"] should return "false". You most likely started the container before you made the settings as I described in part 1. These os.environ variables are important, since the function that lets you pull data from Splunk relies on them. The error in your screenshot "An error occurred: int() argument must be a sting, ..." stems from the fact that the SplunkSearch() function has no values for host/port/token.

import libs.SplunkSearch as SplunkSearch
search = SplunkSearch.SplunkSearch()

You find the source code for the SplunkSearch function in your Jupyter Lab here: /notebooks/libs/SplunkSearch.py. Somewhere in the upper section of this Python code, you see the following.

if "splunk_access_enabled" in os.environ:
   access_enabled = os.environ["splunk_access_enabled"]
      if access_enabled=="1":
         self.host = os.environ["splunk_access_host"]
         self.port = os.environ["splunk_access_port"]
         self.token = os.environ["splunk_access_token"]

As you can see in the code above, the SplunkSearch.py reads the host, port, and token you entered on the settings page if you also set Enable Splunk Access: Yes.

If you are familiar with Splunk's REST API, you recognize that host, port, and token are necessary values to establish a connection from your notebook to Splunk to eventually retrieve search results for your query. I skip the details, but here are a couple lines from SplunkSearch.py that illustrate what packages are used, the connection that is made, as well as the search query that is initiated.

import splunklib.results as splunk_results
import splunklib.client as splunk_client

self._service = splunk_client.connect(host=self.host, port=self.port, token=self.token)

# create a search job in splunk
   job = self.service.jobs.create(
   query_cleaned, 
   earliest_time=earliest, 
   latest_time=latest, 
   adhoc_search_level="smart",
   search_mode="normal")

I hope this helps. Regards,

Gabriel