Getting Data In

Can you help me figure out why some files were not sent to the Heavy forwarder?

kinaba_splunk
Splunk Employee
Splunk Employee

The universal forwarder (UF) seems to read the following files, but the files were not sent to the heavy forwarder (HF) around 11-26-2018 16:16. The following messages appeared in UF's splunkd.log around that time. It seems that Splunk read the files.

11-26-2018 16:16:16.109 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/MMM2018092615.txt 
11-26-2018 16:16:17.882 +0900 INFO TailReader - Batch input finished reading file='/fxxxx/splunk/OOO2018092615.txt

Following messages appear in UF's splunkd.log around the same time.

11-26-2018 16:16:05.005 +0900 INFO TcpOutputProc - Queue for group xxxxx_fwd_intermediate has stopped dropping events 11-26-2018 16:16:10.004 +0900 INFO TailReader - Could not send data to output queue (parsingQueue), retrying... 
11-26-2018 16:16:10.005 +0900 WARN TcpOutputProc - Queue for group xxxxx_fwd_intermediate has begun dropping events 

Could you tell me about solution?

0 Karma

kinaba_splunk
Splunk Employee
Splunk Employee

Please check if the file size is large. For example, the size is 20-180MB, it seems that the default queue size of 500KB is really low for them. So, the queue got full is an expected behavior and increasing the queue size should be a solution for that.

UF's outputs.conf 
[tcpout:xxxxx_fwd_intermediate] 
maxQueueSize = 128MB 

HF's inputs.conf 
[splunktcp://9997] 
disabled = 0 
queueSize = 128MB
0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...