Using Splunk Enterprise 7.3.0 and ITSI 4.3.0
Goal is to configure adaptive thresholding for some metric based KPIs
Using collectd to collect 17 CPU, memory, disk, etc metrics on Unix servers into index=unixperf_metrics
We are unable to see data in the Thresholding section of the KPIs
When choosing a search type of "metric" and choose from the drop downs the index and metric I can see results in the "Thresholding" section and can see overall results in the service
For better search efficiency we created base searches with something like this for the CPU metrics that lists all of the names per each metric type:
| mstats avg(cpu.percent.idle.value) as cpu_idle avg(cpu.percent.interrupt.value) as cpu_interrupt avg(cpu.percent.nice.value) as cpu_nice avg(cpu.percent.softirq.value) as cpu_soft avg(cpu.percent.steal.value) as cpu_steal avg(cpu.percent.system.value) as cpu_system avg(cpu.percent.user.value) as cpu_user WHERE index=unixperf_metrics by host
After creating these base searches and KPI's we receive results in the service but none in the Thresholding section
In opting to remove base search from the equation we selected a search type of base search, same behavior, no results under the Thresholding section however results show in the service, search is:
| mstats avg(cpu.percent.system.value) as cpu_percent WHERE index=unixperf_metrics by host
I think I need to address this to get adaptive thresholding to work, any ideas as to what I'm missing?
