Splunk Search

Send Fields from Search to External Python Script to Update Lookup Table

Contributor

I'm trying to send fields that I gather from a search command and send the results to a external python script. The script will then take those results and update a lookup table. I've gotten the python script working to take 3 arguments and update the csv file. I'm having problems getting the search command to output the information. I thought I was to use the script command, but I guess that is for something else. Has anyone else done this before? Any suggestions?

Thanks in advance.

Tags (3)
0 Karma
1 Solution

Contributor

I went ahead and got the savedsearch working. Now I can just do a ... | table blah1 blah2 blah3 | savedsearchcmd

I had to map the command in the local/commands.conf. The python script uses fileinput to find the text and replace it. Runs like a champ. Thanks again everyone.

Tricky part in python was trying to figure out how it delivered the information.

import splunk.Intersplunk

results = []

try:

results = splunk.Intersplunk.readResults(None, None, True)
for i in results:
    node = i.get('nodes')
    jobid = i.get('jobid')
    status = i.get('typeid')

except:

import traceback

stack = traceback.format_exc()

results = splunk.Intersplunk.generateErrorResults("Error : Traceback: " + str(stack))

View solution in original post

Contributor

I went ahead and got the savedsearch working. Now I can just do a ... | table blah1 blah2 blah3 | savedsearchcmd

I had to map the command in the local/commands.conf. The python script uses fileinput to find the text and replace it. Runs like a champ. Thanks again everyone.

Tricky part in python was trying to figure out how it delivered the information.

import splunk.Intersplunk

results = []

try:

results = splunk.Intersplunk.readResults(None, None, True)
for i in results:
    node = i.get('nodes')
    jobid = i.get('jobid')
    status = i.get('typeid')

except:

import traceback

stack = traceback.format_exc()

results = splunk.Intersplunk.generateErrorResults("Error : Traceback: " + str(stack))

View solution in original post

SplunkTrust
SplunkTrust

Feel free to post your command's definition here if you like.

0 Karma

Path Finder

@wweiland, where are you bringing 'Intersplunk' from?
I could not find it either inside splunk sdk or splunklib libraries.

0 Karma

Contributor

I'm not sure if this still works. This was from 2014 and I'm not in that environment anymore.

0 Karma

SplunkTrust
SplunkTrust

You can do that with native Splunk.

Load new data, append+inputlookup the existing lookup table, run stats by node or whatever you need to merge the two, pipe to outputlookup.

SplunkTrust
SplunkTrust

Take a look at the saved searches in the SoS app, they load stuff and store it in a lookup using outputlookup.

0 Karma

Contributor

martin_mueller, do you possibly have any pointers on what you suggested?

0 Karma

Contributor

New approach. I'll see if I can get that working as well. Thanks for your input.

0 Karma

Contributor

I need something that will update lines in the table. I have a list of compute nodes and when jobs start the job id will be assigned to that node. When the job ends the node will be marked idle. The logs wont produce the states of each node at search time. I also thougt about trying to do a mysql connector, but trying to stick to native splunk if possible.

0 Karma

SplunkTrust
SplunkTrust

Have you considered updating the lookup table directly from the search using outputlookup?

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/outputlookup

0 Karma

Contributor

I added it to the commands.conf, but when I do a search | script.py it fails. Do I have to do the outputResults if I don't plan to return anything?

0 Karma

Champion

is the script working from search bar? then you need to add the script in commands.conf. Need to include

splunk.Intersplunk.outputResults(results)

for output.

http://docs.splunk.com/Documentation/Splunk/6.0.1/AdvancedDev/SearchScripts

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!