Getting Data In

Can you help me figure out why some files were not sent to the Heavy forwarder?

kinaba_splunk
Splunk Employee
Splunk Employee

The universal forwarder (UF) seems to read the following files, but the files were not sent to the heavy forwarder (HF) around 11-26-2018 16:16. The following messages appeared in UF's splunkd.log around that time. It seems that Splunk read the files.

11-26-2018 16:16:16.109 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/MMM2018092615.txt 
11-26-2018 16:16:17.882 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/OOO2018092615.txt

Following messages appear in UF's splunkd.log around the same time.

11-26-2018 16:16:05.005 +0900 INFO TcpOutputProc - Queue for group xxxxx_fwd_intermediate has stopped dropping events 11-26-2018 16:16:10.004 +0900 INFO TailReader - Could not send data to output queue (parsingQueue), retrying... 
11-26-2018 16:16:10.005 +0900 WARN TcpOutputProc - Queue for group xxxxx_fwd_intermediate has begun dropping events 

Could you tell me about solution?

0 Karma

kinaba_splunk
Splunk Employee
Splunk Employee

Please check if the file size is large. For example, the size is 20-180MB, it seems that the default queue size of 500KB is really low for them. So, the queue got full is an expected behavior and increasing the queue size should be a solution for that.

UF's outputs.conf 
[tcpout:xxxxx_fwd_intermediate] 
maxQueueSize = 128MB 

HF's inputs.conf 
[splunktcp://9997] 
disabled = 0 
queueSize = 128MB
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...