Archive
Highlighted

What does it mean for a custom search command to be streamable?

Splunk Employee
Splunk Employee

The commands.conf parameters are not super well-documented online or in the spec file. From the spec file:

streaming = <true/false>
* Is the command streamable.
* Defaults to false.

With streaming=true, does it mean Splunk will invoke the custom search command multiple times as needed during a single execution of a search, whereas, streaming=false means all the events are retrieved, then the custom search script is invoked once?

If that is true, then when streaming=true, does the maxinputs parameter take effect?

Tags (1)
Highlighted

Re: What does it mean for a custom search command to be streamable?

Splunk Employee
Splunk Employee

Here is additional documentation on the topic:

Types of commands

* streaming
 o A streaming command is applied as results travel through the search pipeline.
 o If your script is not streaming, it will only process a single chunk of results. You may 
   specify a search (that contains only streaming commands) to be executed before your  
   non-streaming script, if your script is the very first non-streaming command in the 
   pipeline or if you have 'requires_preop' set to true (it's false by default). 


* streaming
Is your command streamable? streaming = true indicates that your script can be applied  
separately for each chunk of results processed in the search pipeline. Otherwise, your script   
will only be called a single time with all of the input results that it will ever see. 

If your script is not streaming, you may specify a search (that contains only streaming   
commands) to be executed before your script, if your script is the very first non-streaming   
command in the pipeline or if you have requires_preop set to 'true' (it's 'false' by default).

You can find this info here and here

Highlighted

Re: What does it mean for a custom search command to be streamable?

Splunk Employee
Splunk Employee

Thanks, Genti. We saw this link also, should've included it in the description. However, changing the streaming from true to false did not change the behavior--the search still capped at about 10k events when there is well over 50k present. This led us to believe we may have the wrong understanding of streaming.

0 Karma