topic Re: Custom search command -> help in Splunk Search

Custom search command -> help

lpolo — Mon, 28 Sep 2020 11:45:10 GMT

I have the phyton script presented in note 1. How Can I modify this script so it can be called as a splunk search command?

Note 1: uat_qe_feed.py

import urllib2, sys,csv
from lxml import etree
from time import strftime as date

host = ['host1.com','host2.com']

for s in host:

        url = "http://" + s + ":8080/rex/administration?files_used=true"

        try:

                f = urllib2.urlopen(url)
                doc = etree.XML(f.read())
                r = doc.xpath("//str[@name]")


                print(date('%Y-%m-%d %H:%M:%S') + "   " + "qe_host=" + s +  "   " + "Stack=uat" + "   " +  "LOCATION="+ r[1].text + "   " +  "NUMBER="+ r[2].text + "   " + "MAP="+ r[3].text + "   " +  "SET="+ r[4].text)


                f.close()

         except urllib2.URLError, e:

                print(date('%Y-%m-%d %H:%M:%S') + "   " + "qe_host=" + s + "   " + "Stack=uat" + "   " + "Status=QE_Not_in_Service" )

Re: Custom search command -> help

sdaniels — Sat, 28 Apr 2012 11:22:18 GMT

Here are the steps for taking your python script and creating a splunk search command.

http://docs.splunk.com/Documentation/Splunk/latest/developer/searchscripts

Some examples referenced here for you to look at as well:

http://blogs.splunk.com/2011/11/30/using-custom-search-commands-with-splunk-python-sdk/

Re: Custom search command -> help

lpolo — Sat, 28 Apr 2012 12:24:12 GMT

i knew about the links and I was not able to make it work. I just need a single example from the code I presented to have a start up..

Re: Custom search command -> help

sdaniels — Sat, 28 Apr 2012 12:35:48 GMT

What are you trying to accomplish with your custom search command? That may help us get you what you need.

Re: Custom search command -> help

lpolo — Mon, 28 Sep 2020 11:45:12 GMT

Get the result set of an xml file. I have the script I presented in the initial post. I am able to print the results. I need to get these results from splunk by executing the script as a custom search. For example:

|uat_qe_feed

The result of this search command is the result of the last line of my phyton script:

print(date('%Y-%m-%d %H:%M:%S') + " " + "qe_host=" + s + " " + "Stack=uat" + " " + "LOCATION="+ r[1].text + " " + "NUMBER="+ r[2].text + " " + "MAP="+ r[3].text + " " + "SET="+ r[4].text)

What should I add in my code so this script can be called as a search command

Re: Custom search command -> help

lpolo — Sat, 28 Apr 2012 17:19:55 GMT

I am getting the result set from a REST API call as shown in the script I presented. I am not indexing the result set.
I just need to know from this script example, how to converted to a custom search command.

Re: Custom search command -> help

gkanapathy — Sun, 29 Apr 2012 18:32:40 GMT

if you're looking for examples, several of the shipped Splunk search commands are in fact Python scripts. Look in $SPLUNK_HOME/etc/apps/search/bin and $SPLUNK_HOME/etc/apps/search/default/commands.conf.

Re: Custom search command -> help

gkanapathy — Sun, 29 Apr 2012 18:41:23 GMT

I suspect you are fundamentally misunderstanding what a search command is good for. You appear to be trying to feed raw data into Splunk. Normally you would do this via a scripted input or simply a file, and index the data. But since a custom search command can run arbitrary code, it's expected that it outputs CSV field data. You can certainly just pass in raw text by putting it into a _raw CSV field, but it would make more sense if you also, at minimum, included _time in epoch time, as well as the other fields you already have available in Python. By putting it back into raw text line, it's wasteful, as you're simply forcing Splunk to re-parse fields that you've already parsed out.

Re: Custom search command -> help

lpolo — Mon, 30 Apr 2012 11:49:37 GMT

Thanks.

Solved by using as example:

/opt/splunk/etc/apps/search/bin/google.py

Re: Custom search command -> help

lpolo — Mon, 30 Apr 2012 14:29:44 GMT

http://splunk-base.splunk.com/answers/7909/where-is-the-python-api-for-splunkintersplunk

Re: Custom search command -> help

lpolo — Tue, 01 May 2012 13:41:45 GMT

Script example customsearch.py:

import urllib2, sys,csv,time
from lxml import etree
import splunk.Intersplunk as si

host = ['host1.net','host2.net']
for s in host:

        url = "http://" + s + ":8080/rex/administration?files_used=true"
        #print(url)
        results = []
        now = str(int(time.mktime(time.gmtime())))

        try:
                f = urllib2.urlopen(url)
                doc = etree.XML(f.read())


                one =  doc.xpath("//str[@name='LINEUP']/text()")
                two  = doc.xpath("//str[@name='LINEAR']/text()")
                three  = doc.xpath("//str[@name='BITSET']/text()")

                results.append({'_time' : now,'qe_host' : s,'Stack' : 'cim','Status' : 'up','LINEUP' : one,'LINEAR' :  two,'BITSET' :  three})

                si.outputResults(results)



        except urllib2.URLError, e:
                results.append({'_time' : 'now','qe_host' : s,'Stack' : 'cim','Status' : 'down','LINEUP' : '','LINEAR' :  '','BITSET' :  ''})
                si.outputResults(results)

        f.close()

commands.conf
[customsearch]
filename = customsearch.py
generating = true
maxinputs = 1