splunkd's processing queues are full in Heavy Forwarder

[Eshwar]

Hi Experts,

We have recently installed Heavy Forwarder and disabled the indexing on it and also we are not forwarding any data from forwarders as of now but all the queue are full in HWF. Don't understand how HWF is full simply without getting any data. Please suggest how to clear them and make it as normal.

Regards,

Eshwar

[richgalloway]

Are you sure the HF is not forwarding any data?  By default, it will send its own logs.  btool can show what inputs are enabled.

splunk btool inputs list --debug

The fix is to remove whatever is blocking the queues.  In this case, make sure the HF has indexers to send to.

[Eshwar]

Hi @richgalloway ,

 We are forwarding the data to Cloud instance from HWF but we don't see any data on Cloud instance. Can you suggest how to remove the blocking queues in HWF as my understand disable is the option right?

Regards,

Eshwar

[richgalloway]

The HF should be logging messages about why it can't send to Splunk Cloud.  Please share those messages so we can suggest solutions.  Once that is resolved, the queues will decrease.

Confirm your network allows connections from the HF to your Splunk Cloud indexers.

Verify you have installed the "Universal Forwarder" app from your Splunk Cloud instance on the HF.  Yes, an app called "Universal Forwarder" really does go on a Heavy Forwarder.

Disabling inputs will prevent more data from being added to the queues, but will not clear the queues.  Restarting the HF will clear the in-memory queues, however.