Monitoring Splunk

piping in splunk

nowakdaw
Path Finder

Hello All,

Does anyone know how piping in splunk is performed. I tried to search for information on this subject but unfortunately I am unable to find anything on it. My question is: does it take the search results from the buffer and then searches on it when piping is done.

To clarify if I search for host="some_host" | source="testing_source" does splunk first search for some host and then from that buffer searches for the source testing_source on it. OR does it search for some_host and then when you pipe it searches again from the entire buffer?

The main purpose of this question is performance.

Thank you for all your help!

Tags (3)
1 Solution

piebob
Splunk Employee
Splunk Employee

this article in the documentation provides an overview of how the search pipeline works:

http://docs.splunk.com/Documentation/Splunk/latest/User/HowSearchCommandsWork

here is a relevant snippet:

"The "search pipeline" refers to the structure of a Splunk search, in which consecutive commands are chained together using a pipe character that tells Splunk to use the output or result of one command as the input for the next command."

your first interpretation is correct--the goal here is to filter down your results set as much as possible before performing calculations or other actions on the final set of results.

View solution in original post

vdeshpandegrp
New Member

What if I don't want to pipe my results, i.e I want each eval to be performed on the entire buffer and not just the subset?

For example:
eval successful_transitions = case(searchmatch("CASE(ActiveSuccesses)"),"active",searchmatch("CASE(InactiveSuccesses)"),"inactive")
| stats count as successes by successful_transitions
| eval failed_transitions = case(searchmatch("[active-failure]"),"active",searchmatch("[inactive-failure]"),"inactive")
| stats count as failures by failed_transitions

Here I want to find, of all the events, How many events are active/inactive successful and how many are active/inactive failed??

Thanks

0 Karma

nowakdaw
Path Finder

Yes! I apologize for my carelessness. Thank you for pointing that out.

0 Karma

Ayn
Legend

Note that your pipe example is syntactically incorrect - you need a command after the pipe. What you've done is added another search filter after the pipe. This filter should be part of the search command before the pipe instead.

piebob
Splunk Employee
Splunk Employee

this article in the documentation provides an overview of how the search pipeline works:

http://docs.splunk.com/Documentation/Splunk/latest/User/HowSearchCommandsWork

here is a relevant snippet:

"The "search pipeline" refers to the structure of a Splunk search, in which consecutive commands are chained together using a pipe character that tells Splunk to use the output or result of one command as the input for the next command."

your first interpretation is correct--the goal here is to filter down your results set as much as possible before performing calculations or other actions on the final set of results.

yannK
Splunk Employee
Splunk Employee

"|" This is a pipe

alt text

ChrisG
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...