All Apps and Add-ons

How the heavy forwarder and Splunk DB connect work when indexers is stopping for these upgrading.

Path Finder

Hi Splunk Professionals,

I am going to upgrade my splunk components.
Along with upgradeing, I am wondering what is the best way to prevent from losing the DB log when stopping indexers.

My enviroment is the below.
- Deploying indexer cluster (3 indexers)
- "Splunk DB connect App(v3.x) is working in Heavy Forwarder. And Heavy Forwarder is monitoring DB logs constantly and forwarding indexers with load balancing.

I am concerned how the heavy forwarder and Splunk DB connect work when indexers is stopping for these upgrading.

In my opinion, I think heavy forwarder will not drop DB logs when stopping indexers , because heavy forwarder is holding wait queues.
While I have no idea about how Splunk DB connect work when wait queues reach the max value, because its inputs setting access DB continuously.

Does anyone have any tips to prevent dropping DB logs while indexer is upgrading?
Or Does anyone know how the Splunk DB connect work when indexers is stopping?
Is there the case that DB logs is dropped cause of Splunk DB connect input setting to send queries?

I will appreciate your any advice and comment.

Best regardes,

0 Karma

Splunk Employee
Splunk Employee

Splunk generally works like a pipeline. You can submit formally to support for validation, but my understanding is that data stops getting forwarded and when the data out pipeline fill on the HF the mod inputs would stop collecting new data. Or maybe I'm being too optimistic.

0 Karma

Ultra Champion
  1. why would you stop / upgrade all 3 indexers at the exact same time?
  2. you can increase your persistent queue on HF
  3. if you using a rising column, you can always stop the input and get back to that particular column
  4. if you are using batch, stop the input and enable back when indexers are up
  5. if you have many inputs, see item #1
0 Karma

Path Finder

Hi adonio,

I appreciate your much types of tips.
I have both inputs settings, but not much.
So I will let batch setting disabled and set higher value of persistentQueueSize.

I am curious how Splunk DB connect inputs setting work when the persistentQueueSize of forwarder reaches max value.

I have read the doc and understood that forwarder will stop sending data when the persistentQueueSize of forwarder reaches max value.

However there is no description about input setting.
Does Splunk DB connect input setting (like batch) run and keep requesting query to DB server continuously, even if forwarder stop sending data ?


0 Karma


Didn't get the point, what is the need to persistent queue in this case. it is better to upgrade indexer one by one. And if multi-site cluster then site by site.

If one indexer is down, the data will load balance to other automatically.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...