Deployment Architecture

Is there a way to natively query a web service?

Builder

I have searched through splunk-base and found some answers on how to query a web service, e.g. by creating a script with curl, or by using an app like webmon.

I have a web service that returns a value, how would I go about incorporating this into an eval statement? e.g.

... | eval some_data=url_fetch("http://my-server/my-endpoint?someParam=".ExtractedField, "Accept: text/plain") | table ExtractedField some_data

So I want to send an extracted field to a service and get the response back (possibly even parsing it). I am guessing the best way to do this would be to write an app to cater for the specifics of this (anyone know of an app that provides this sort of function?).

thanks.

Tags (3)
1 Solution

Legend

Best option would probably be to write your own custom search command. There are some search commands that come with Splunk out of the box that could serve as some inspiration, for instance the google command (located in $SPLUNK_HOME/etc/apps/search/bin/google.py) that performs a Google search for you and shows the results.

There is a good docs section on writing and using custom search commands, see here: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Aboutcustomsearchcommands

View solution in original post

Builder

Ok... So i have written a custom search command that makes use of urllib2 to query the web service. Now the simple part that is proving challenging 🙂

mycustomsearch.py (simplified):

results = []
req = urllib2.urlopen('http://my-server/endpoint')
res = req.read()
results.append({'_time' : now, 'ResultField' : res})
splunk.Intersplunk.outputResults(results[1])

So now I can run | mycustomsearch "param1" and I get tabulated results:

_time    ResultField
<time>   "foobar"

I have a field extractor for ResultField, so I am trying to filter by that field:

eventtype="SomeEvent" [ mycustomsearch "param" | fields + ResultsField ]

Assuming the above would filter SomeEvent with ResultsField = "foobbar", but I am not able to get this. How could I filter by the output of the custom command?

0 Karma

Builder

just had a look on the splunk server i was testing it - we never implemented it and so removed the code - don't have the actual code available, but used the google.py code when I wrote it as a template.

0 Karma

Builder

Could you post your entire code ? I'm also interested in a webservice commmand...

0 Karma

Builder

yeah, it is a typo in this post, i have consistency in my searches in splunk.

the format option works great, thanks.

0 Karma

Legend

You could check which specific output you get from that subsearch by running it by itself instead and appending "| format" at the end. I notice that the table you show as output from your custom search command has the output fieldname "ResultField", whereas the subsearch returns the field "Resultsfield" - I'm assuming it's a typo but wanted to point that out just in case.

Legend

Best option would probably be to write your own custom search command. There are some search commands that come with Splunk out of the box that could serve as some inspiration, for instance the google command (located in $SPLUNK_HOME/etc/apps/search/bin/google.py) that performs a Google search for you and shows the results.

There is a good docs section on writing and using custom search commands, see here: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Aboutcustomsearchcommands

View solution in original post

Legend

You can use it exactly like brettcave wrote.

0 Karma

Explorer

this doesnt work mate. gives an error. any ideas on how to use this command?

0 Karma

Builder

never mind.

| google "term"
0 Karma

Builder

Is there documentation on how to use the google command? Googling for it or searching through splunk-base isn't giving any results 😞

0 Karma