topic Re: How to create pre-actions to a custom StreamingCommand with SDK v2? in Splunk Dev

How to create pre-actions to a custom StreamingCommand with SDK v2?

davietch — Thu, 31 Jan 2019 10:50:48 GMT

Hi,

I am trying to build a new custom StreamingCommand with SDK v2.

I would like this command to create a unique folder when it starts. The folder name is randomly generated with python.
The issue is that the streaming command calls these few lines of code (where I create a random folder) several times along the time the command run.
This means that instead of having just one folder, the command can create hundred of folder if the command run for a long time...

I think it is the normal behavior of a streaming command where it is called on a set of events. But is there a way to run some part of the code just once, at the beginning of the search job?

I tried to place my code outside of the

def stream(self, records):

and right before :

dispatch(MyCommand, sys.argv, sys.stdin, sys.stdout, __name__)

But it is not changing anything.

Any idea?

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

jkat54 — Thu, 31 Jan 2019 12:36:04 GMT

You need to change your command to not allow it to run in preview. There’s a setting in commands.conf to disable preview for the spl command.

With preview on, chunks of data go through your command as they arrive causing your code to execute multiple times. With preview off, your command only executes once.

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

jkat54 — Thu, 31 Jan 2019 12:37:06 GMT

run_in_preview = [true|false]
* Specify whether to run this command if generating results just for preview
  rather than final output.
* Defaults to true

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

davietch — Thu, 31 Jan 2019 12:50:40 GMT

Hi Jkat,

Thank you but I already disabled it:

run_in_preview = false

Actually, this solved one issue:
When running a "stats count by X", and then applied my custom command, I was received the preview data. Turning this setting to false forced Splunk to give me the final result for each X field.
The problem is that it does not block Splunk from running several times the "stream" function...

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

jkat54 — Thu, 31 Jan 2019 12:56:37 GMT

Can you share your commands.conf?

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

davietch — Thu, 31 Jan 2019 13:13:50 GMT

Sure:

[savetofile]
filename = savetofile.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
run_in_preview = false

Re: How to create pre-actions to a custom StreamingCommand with SDK v2?

jkat54 — Fri, 01 Feb 2019 02:17:53 GMT

I’ve built everything using intersplunk.py so far. I don’t have much experience with v2 of the search sdk. I will say I think you have to have chunked=true in commands.conf for v2 commands. Not sure what else could cause it from a settings stand point.

Might have to share your code so we can help more.