I am going to upgrade my splunk components.
Along with upgradeing, I am wondering what is the best way to prevent from losing the DB log when stopping indexers.
My enviroment is the below.
- Deploying indexer cluster (3 indexers)
- "Splunk DB connect App(v3.x) is working in Heavy Forwarder. And Heavy Forwarder is monitoring DB logs constantly and forwarding indexers with load balancing.
I am concerned how the heavy forwarder and Splunk DB connect work when indexers is stopping for these upgrading.
In my opinion, I think heavy forwarder will not drop DB logs when stopping indexers , because heavy forwarder is holding wait queues.
While I have no idea about how Splunk DB connect work when wait queues reach the max value, because its inputs setting access DB continuously.
Does anyone have any tips to prevent dropping DB logs while indexer is upgrading?
Or Does anyone know how the Splunk DB connect work when indexers is stopping?
Is there the case that DB logs is dropped cause of Splunk DB connect input setting to send queries?
Splunk generally works like a pipeline. You can submit formally to support for validation, but my understanding is that data stops getting forwarded and when the data out pipeline fill on the HF the mod inputs would stop collecting new data. Or maybe I'm being too optimistic.