I deployed Splunk Enterprise edition 7.2.3 and gave it 1 TB data for indexing. The data is available locally. Initially, when the queues(parsing, merging, typing and indexing) are empty, I am getting an index rate of ~2MB per second. But as the queues get filled, the indexing rate drops to a few KBs per second. But from there on the indexing rate keeps increasing and dropping. Also, sometimes, indexing doesn't happen at all when the parsing queue and merging queue are full.
Is this behavior expected?
How do we achieve consistent indexing rate?
Is increasing parsing queue size a solution? But that will also get filled soon.
Note: 20GB license is also added.
... View more