Howto get a list of ITSI services with source of KPI base search

krusovice — Mon, 29 Apr 2019 05:27:00 GMT

Hello,

In my environment, I have a long list of ITSI services (created by someone else) which using default KPI base search. These default KPI base search is running every mins for 1 min data and it has causes some impact to the indexers.

Without going through the UI for ITSI services and checking the search source for each KPI, is there any query I can use to generate the list of the ITSI services + KPI base search + base search source name?

Thanks.

Re: Howto get a list of ITSI services with source of KPI base search

jaime_ramirez — Wed, 17 Jul 2019 20:59:40 GMT

There is this lookup included in ITSI that includes all info for each KPI defined:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name

If you want it separated:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name kpis.* as kpis_*
| eval kpis_info = mvzip(mvzip(mvzip(kpis__key, kpis_base_search, "==@@=="), kpis_search_type, "==@@=="), kpis_title, "==@@==")
| fields service_name service_id kpis_info
| mvexpand kpis_info
| eval kpis_info=split(kpis_info, "==@@==")
| eval kpis__key=mvindex(kpis_info, 0)
| eval kpis_base_search=mvindex(kpis_info, 1)
| eval kpis_search_type=mvindex(kpis_info, 2)
| eval kpis_title=mvindex(kpis_info, 3)
| fields - kpis_info

You could also try searching the splunk scheduler log or a version of the KPI Performance Panel included in the ITSI Healthcheck Dashboard:

index=_internal sourcetype=scheduler savedsearch_name="Indicator*"
            | stats count as run_count, count(eval(status!="success")) as failed_count, count(eval(suppressed!="0")) as suppressed_count,
            avg(run_time) as avg_runtime, max(run_time) as max_runtime, earliest(_time) as first, latest(_time) as last
            by savedsearch_name
            | eval KPI_search_type=if(savedsearch_name like "%Shared%", "base", "ad hoc")
            | eval runtime_headroom_pct=round((100-(max_runtime/((last-first)/(run_count-1))*100)),1)
            | eval avg_runtime=round(avg_runtime, 2)
            | eval max_runtime=round(max_runtime, 2)
            | table savedsearch_name KPI_search_type failed_count suppressed_count runtime_headroom_pct avg_runtime max_runtime run_count
            | sort +runtime_headroom_pct

Hope it helps. Cheers!!!

Re: Howto get a list of ITSI services with source of KPI base search

a1bg503461 — Wed, 15 Feb 2023 15:13:19 GMT

I want to download the service_kpi_sbs_lookup.csv file and import it in another test splunk whit Lookup Editor. Does anyone know where is service_kpi_sbs_lookup.csv under linux ? Thanks

topic Re: Howto get a list of ITSI services with source of KPI base search in Splunk ITSI

Howto get a list of ITSI services with source of KPI base search

Re: Howto get a list of ITSI services with source of KPI base search

Re: Howto get a list of ITSI services with source of KPI base search