Getting Data In

Which input provides better performance batch or monitor

Communicator

I'm looking to forward data collected via a lightweight forwarder. Which input provides better performance batch or monitor? I'm trying to reduce the disk footprint but I'd like to get the data to the indexer as quickly as possible.

0 Karma
1 Solution

Splunk Employee
Splunk Employee

Batch and monitor are both delivered by the exact same subsystem within Splunk, so there shouldn't be any significant difference in performance.

View solution in original post

Splunk Employee
Splunk Employee

Batch and monitor are both delivered by the exact same subsystem within Splunk, so there shouldn't be any significant difference in performance.

View solution in original post

Contributor

Is this 20MB value tunable? I would like to have a forwarder reading from many files and fan them out to many indexers as fast as it can. The single threaded nature is killing me.

0 Karma

Splunk Employee
Splunk Employee

Both batch and monitoring single thread reading of files that have more than 20MB remaining to read. This behavior has a positive impact on performance since reading is fundamentally very fast, but parsing, which is often single threaded is the main bottleneck and performs better with coherent streams of data.

Communicator

Stephen, does the batch input eat files one at a time? If it does I'd expect the file monitor to perform better?

0 Karma