Is there any Splunk query that would provide the details of HF ports where incoming logs are dropping?
For ex I have one HF. Now I want to know if there are any UDP ports where incoming logs are dropping and the logs are not indexing in splunk. I can perform tcpdump to get this. But I want to know the historical details from when this has been started, how many ports are involved in the past in such log dropping incident etc. So it would be better if splunk can capture these events and show us the details of such events. Is there any facilities in Splunk?
If the queue's blocking it'll drop traffic - index=_internal Metrics blocked=true NOT StreamedSearch | table _time, host, name, max_size_kb, current_size_kb is a starter search for showing where Splunk knows it's blocking.
To monitor UDP queue headroom, I use index=_internal Metrics group=queue NOT StreamedSearch name=udp* | eval headroom=max_size_kb-current_size_kb | timechart avg(headroom) by host