Which input provides better performance batch or monitor

[Marinus]

I'm looking to forward data collected via a lightweight forwarder. Which input provides better performance batch or monitor? I'm trying to reduce the disk footprint but I'd like to get the data to the indexer as quickly as possible.

[Stephen_Sorkin · Accepted solution]

Batch and monitor are both delivered by the exact same subsystem within Splunk, so there shouldn't be any significant difference in performance.

[vbumgarner]

Is this 20MB value tunable? I would like to have a forwarder reading from many files and fan them out to many indexers as fast as it can. The single threaded nature is killing me.

[Stephen_Sorkin]

Both batch and monitoring single thread reading of files that have more than 20MB remaining to read. This behavior has a positive impact on performance since reading is fundamentally very fast, but parsing, which is often single threaded is the main bottleneck and performs better with coherent streams of data.

[Marinus]

Stephen, does the batch input eat files one at a time? If it does I'd expect the file monitor to perform better?