I'm trying to use data from a search in a custom command.
source | scrapy url=uri
This gives me the following error:
Error in 'scrapy' command: This command must be the first command of a search.
It works when I use it as follows:
| scrapy url="www.splunk.com"
How can I make it work in combination with my search index?
the important thing is to import the
splunk.Intersplunk module in your script:
and read the results from the search into your script:
myresults,dummyresults,settings = splunk.Intersplunk.getOrganizedResults() # getting search results form Splunk for r in myresults: # loop the results
This way your script can pick up fields from the previous search results and it should work as expected if your search results contain a field called
url (just rename
url) or change your script to use
uri instead of
The link posted by @somesoni2 provides useful information as well.
Hope this helps ...
Thanks MuS for your awnser,
I can't seem to get this right. Could you be so kind to help me out with an easy example?
import splunk.Intersplunk # GET DATA FROM SEARCH myresults,dummyresults,settings = splunk.Intersplunk.getOrganizedResults() # ILL DO SOME FANCY STUFF HERE # SEND DATA BACK
Thanks for helping out here.
Take a look at the docs, as usual everything you need is in there 😉
This http://docs.splunk.com/Documentation/Splunk/6.4.2/Search/Customsearchcommandshape will show an excellent example of a custom command which will use the previous search results, do stuff with it and return something to Splunk.
Since your second search (
| scrapy url="www.splunk.com")works, It seems like you've a custom "Generating" search command which is generating the results.
For a custom search command to work with your first syntax (
source | scrapy url=uri), it should be a non-generating command. See definitions of customer search commands here