Events Being Passed to Custom Commands More Than Once

[cschmidt0121]

I've been working with custom commands for a while, and I've noticed some weird behavior regarding how events are passed into commands. It is my understanding that regardless of whether a command is declared to be streaming or not, Splunk may choose to send events to the command in chunks instead of one large set. So what I would expect to see is something like this:

For a total of 15,000 events piped into custom command "cmd"...

Splunk sends cmd events 0-5000
cmd completes and outputs results

Splunk sends cmd events 5001-10000
cmd completes and outputs results

Splunk sends cmd events 10001-15000
cmd completes and outputs results

However, by having my command log the events it receives, I instead see something that resembles this:

Splunk sends cmd events 0-5000
cmd completes and outputs results

Splunk sends cmd events 0-10000
cmd completes and outputs results (these results OVERWRITE previous results!)

Splunk sends cmd events 0-15000
cmd completes and outputs results (these results OVERWRITE previous results!)

So events 0-5000 end up being passed to the command 3 times, and events 5001-10000 are passed twice. Is this intended behavior? If so, can anyone explain why?

[kchen_splunk]

Setting "run_in_preview = false" in commands.conf will solve the problme

[dart]

This is, as far as I know the intended behaviour for preview functionality in Splunk.

Overwriting previous results makes sense to me - for example if you were implemented a command that added some global state to each event (like eventstats) then the more complete input would be correct, and the partial one doesn't.

I'd suggest also that you have a look at the new python SDK examples which make writing custom search commands easier

[cschmidt0121]

Ooooh. I see. Thank you very much for the response.

[dart]

It's so we can display the preview results - it's pretty handy for seeing if you're going to get the right data.

[cschmidt0121]

Yeah that certainly makes sense. I'm still confused as to why Splunk runs the command multiple times in the first place, though, if it's just going to discard the results of the first two runs.

Thanks for the link to the new SDK examples; I'm looking over them now.