hello everyone .forgive me for not being good at English.
I encountered a problem today, A sourcetype ( sourcetype=example1 ) log was suddenly lost. My forwarding mode is as follows:
universal forwarder—>heavy forwarder—>indexer cluster
On the host with a universal forwarder, uf monitored 2 log files（corresponding to 2 sourcetypes（example1 and example2） ）.
Why example1's logs Unable to send to HF, but example2's logs are successfully sent to HF and then to the indexer?
I checked my UF first. I saw this message in the splunk.log of UF.
Could not send data to output queue(parsingQueue),retrying.....
Then I look at metrics.log on HF. I saw that indexequeue, typingqueue, aggqueue and splunktcpin were blocked.
I have 3 questions about this.
1、Why is the queue suddenly blocked? I checked the monitoring history of zabbix, The HF host resources are always idle.Usually, what causes the queue to be blocked?
2、When I restart Universal forwarder, everything is back to normal.Why do I just restart the UF and the queue is working ?
3、What are the good ways to find out why the queue is blocked?
