Getting Data In

Which input provides better performance batch or monitor

Marinus
Communicator

I'm looking to forward data collected via a lightweight forwarder. Which input provides better performance batch or monitor? I'm trying to reduce the disk footprint but I'd like to get the data to the indexer as quickly as possible.

0 Karma
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

Batch and monitor are both delivered by the exact same subsystem within Splunk, so there shouldn't be any significant difference in performance.

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

Batch and monitor are both delivered by the exact same subsystem within Splunk, so there shouldn't be any significant difference in performance.

vbumgarner
Contributor

Is this 20MB value tunable? I would like to have a forwarder reading from many files and fan them out to many indexers as fast as it can. The single threaded nature is killing me.

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

Both batch and monitoring single thread reading of files that have more than 20MB remaining to read. This behavior has a positive impact on performance since reading is fundamentally very fast, but parsing, which is often single threaded is the main bottleneck and performs better with coherent streams of data.

Marinus
Communicator

Stephen, does the batch input eat files one at a time? If it does I'd expect the file monitor to perform better?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...