Hi @hazem , it depends on how many logs you have to transmit: e.g. a Domain Controller has to transmit more logs than a server, if you have application logs they must consider them. Anyway, between...
See more...
Hi @hazem , it depends on how many logs you have to transmit: e.g. a Domain Controller has to transmit more logs than a server, if you have application logs they must consider them. Anyway, between intermediate UFs and Indexer, I hint to avoid limits. You can configure the max throughtput suing the maxKBps parameter on the UFs. My hint is to leave the default values, changing only maxKBps for intermediate UFs, and analyzing both if you have netweork congestions and your Indexers can index all logs with an acceptable delay. Another analysis to perform is the presence of queues, using this search: index=_internal source=*metrics.log sourcetype=splunkd group=queue
| eval name=case(name=="aggqueue","2 - Aggregation Queue",
name=="indexqueue", "4 - Indexing Queue",
name=="parsingqueue", "1 - Parsing Queue",
name=="typingqueue", "3 - Typing Queue",
name=="splunktcpin", "0 - TCP In Queue",
name=="tcpin_cooked_pqueue", "0 - TCP In Queue")
| eval max=if(isnotnull(max_size_kb),max_size_kb,max_size)
| eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size)
| eval fill_perc=round((curr/max)*100,2)
| bin _time span=1m
| stats Median(fill_perc) AS "fill_percentage" perc90(fill_perc) AS "90_perc" max(max) AS max max(curr) AS curr by host, _time, name
| where (fill_percentage>70 AND name!="4 - Indexing Queue") OR (fill_percentage>70 AND name="4 - Indexing Queue")
| sort -_time if you have queues, you can modify the maxSize parameter for the queues and the maxKBps. Ciao. Giuseppe