topic Re: piping in splunk in Monitoring Splunk

piping in splunk

nowakdaw — Mon, 28 Sep 2020 12:27:36 GMT

Hello All,

Does anyone know how piping in splunk is performed. I tried to search for information on this subject but unfortunately I am unable to find anything on it. My question is: does it take the search results from the buffer and then searches on it when piping is done.

To clarify if I search for host="some_host" | source="testing_source" does splunk first search for some host and then from that buffer searches for the source testing_source on it. OR does it search for some_host and then when you pipe it searches again from the entire buffer?

The main purpose of this question is performance.

Thank you for all your help!

Re: piping in splunk

piebob — Tue, 18 Sep 2012 17:20:39 GMT

this article in the documentation provides an overview of how the search pipeline works:

http://docs.splunk.com/Documentation/Splunk/latest/User/HowSearchCommandsWork

here is a relevant snippet:

"The "search pipeline" refers to the structure of a Splunk search, in which consecutive commands are chained together using a pipe character that tells Splunk to use the output or result of one command as the input for the next command."

your first interpretation is correct--the goal here is to filter down your results set as much as possible before performing calculations or other actions on the final set of results.

Re: piping in splunk

yannK — Tue, 18 Sep 2012 17:25:09 GMT

"|" This is a pipe

Re: piping in splunk

Ayn — Tue, 18 Sep 2012 18:21:25 GMT

Note that your pipe example is syntactically incorrect - you need a command after the pipe. What you've done is added another search filter after the pipe. This filter should be part of the search command before the pipe instead.

Re: piping in splunk

nowakdaw — Tue, 18 Sep 2012 18:27:46 GMT

Yes! I apologize for my carelessness. Thank you for pointing that out.

Re: piping in splunk

ChrisG — Sat, 05 Dec 2015 17:16:13 GMT

Current doc link: http://docs.splunk.com/Documentation/Splunk/6.3.1/Search/Aboutsearchlanguagesyntax#About_the_search_pipeline

Re: piping in splunk

vdeshpandegrp — Tue, 29 Sep 2020 17:24:27 GMT

What if I don't want to pipe my results, i.e I want each eval to be performed on the entire buffer and not just the subset?

For example:
eval successful_transitions = case(searchmatch("CASE(ActiveSuccesses)"),"active",searchmatch("CASE(InactiveSuccesses)"),"inactive")
| stats count as successes by successful_transitions
| eval failed_transitions = case(searchmatch("[active-failure]"),"active",searchmatch("[inactive-failure]"),"inactive")
| stats count as failures by failed_transitions

Here I want to find, of all the events, How many events are active/inactive successful and how many are active/inactive failed??

Thanks