topic Re: Help with custom search command (wait) needed in Splunk Search

Help with custom search command (wait) needed

damucka — Fri, 09 Aug 2019 13:12:58 GMT

Hello,

I need to apply 60 sec delay between two SPL commands, which start and collect the DB trace per dbxquery.
In between, there should be 60 sec time. As I did not find anything corresponding, I wrote the simplest possible .py script:

import time
time.sleep(60)

and call it like | sleep60. It waits 60 sec, but then throws an error:

08-09-2019 15:05:56.331 INFO  UserManager - Unwound user context: d038423 -> NULL
08-09-2019 15:05:56.332 INFO  PipelineComponent - Process delayed by 59.062 seconds, perhaps system was suspended?
08-09-2019 15:05:56.332 ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Error in 'sleep60' command: External search command exited unexpectedly.

Now I am thinking that the .py will not be as easy as that perhaps. Of course I expect that all the variables from before the command execution will still be visible, etc.
Any idea why I get this error and how a simple delay script should look like?

Kind Regards,
Kamil

Re: Help with custom search command (wait) needed

DavidHourani — Fri, 09 Aug 2019 14:59:33 GMT

Hi @damucka,

You're always doing cool stuff!

Try making the .py append a column to the results saying "60 seconds ok" That would make your debugging way easier.

Re: Help with custom search command (wait) needed

DavidHourani — Sat, 10 Aug 2019 07:35:56 GMT

Any updates Kamil ?

Re: Help with custom search command (wait) needed

damucka — Sat, 10 Aug 2019 14:30:51 GMT

Hi David,

If you mean that in my python code I should do sth to append the result, then I am afraid I would ask for help with this. I mean I can start googling, but you would know for sure how this should go. You can see on the above example of my .py how simple it is and if I now have to import pip splunk-sdk, use the input / output classes to produce the result in python ... I am not sure if I want to begin with this unless it is absolutely necessary.
Or you would just tell me how the code should look like :-).
Unless you mean the appendcols in SPL, from where my sleep60 is called, but this I would not know how it should help here either.
Sorry for my ignorance.

Regards,
Kamil

Re: Help with custom search command (wait) needed

DavidHourani — Sat, 10 Aug 2019 16:10:45 GMT

Hey Kamil,

Have a look here :
https://docs.splunk.com/Documentation/Splunk/7.3.1/Search/Customsearchcommandshape
It's a good example that shows you how a custom command takes and input and applies a logic to it.

In your case your logic should be :
1-Data goes in.
2-Wait 60 seconds.
3-output the same data (+ comment to indicate wait is complete--- optional).
In your case above in the question your output gives nothing, hence the error.

Re: Help with custom search command (wait) needed

damucka — Wed, 14 Aug 2019 06:36:18 GMT

Hello David,

sorry, it took a while because we had some issues with our python 2.7 installation, where we were not able to get the splunk-sdk running so we had to workaround it pointing to the python 3.5.
The code for waiting command is quite simple, googled and assembled from Splunk examples :-).
Please see below.

Kind Regards,
Kamil

import sys
sys.path.append("/usr/local/lib/python3.5/site-packages/splunk_sdk-1.6.6-py2.7.egg")
import time
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration()
class MyCommand(StreamingCommand):
        def stream(self,records):
                time.sleep(60)
                for record in records:
                        yield record

if __name__ == "__main__":
        dispatch(MyCommand, sys.argv, sys.stdin, sys.stdout, __name__)

Re: Help with custom search command (wait) needed

DavidHourani — Wed, 14 Aug 2019 06:43:36 GMT

Awesome ! Great work Kamil 😉

Re: Help with custom search command (wait) needed

DavidHourani — Wed, 14 Aug 2019 06:52:30 GMT

PS: Please accept the answer to close this post.