Splunk IT Service Intelligence

Howto get a list of ITSI services with source of KPI base search

krusovice
Path Finder

Hello,

In my environment, I have a long list of ITSI services (created by someone else) which using default KPI base search. These default KPI base search is running every mins for 1 min data and it has causes some impact to the indexers.

Without going through the UI for ITSI services and checking the search source for each KPI, is there any query I can use to generate the list of the ITSI services + KPI base search + base search source name?

Thanks.

Tags (1)
1 Solution

jaime_ramirez
Communicator

Hi

There is this lookup included in ITSI that includes all info for each KPI defined:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name

If you want it separated:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name kpis.* as kpis_*
| eval kpis_info = mvzip(mvzip(mvzip(kpis__key, kpis_base_search, "==@@=="), kpis_search_type, "==@@=="), kpis_title, "==@@==")
| fields service_name service_id kpis_info
| mvexpand kpis_info
| eval kpis_info=split(kpis_info, "==@@==")
| eval kpis__key=mvindex(kpis_info, 0)
| eval kpis_base_search=mvindex(kpis_info, 1)
| eval kpis_search_type=mvindex(kpis_info, 2)
| eval kpis_title=mvindex(kpis_info, 3)
| fields - kpis_info

You could also try searching the splunk scheduler log or a version of the KPI Performance Panel included in the ITSI Healthcheck Dashboard:

index=_internal sourcetype=scheduler savedsearch_name="Indicator*"
            | stats count as run_count, count(eval(status!="success")) as failed_count, count(eval(suppressed!="0")) as suppressed_count,
            avg(run_time) as avg_runtime, max(run_time) as max_runtime, earliest(_time) as first, latest(_time) as last
            by savedsearch_name
            | eval KPI_search_type=if(savedsearch_name like "%Shared%", "base", "ad hoc")
            | eval runtime_headroom_pct=round((100-(max_runtime/((last-first)/(run_count-1))*100)),1)
            | eval avg_runtime=round(avg_runtime, 2)
            | eval max_runtime=round(max_runtime, 2)
            | table savedsearch_name KPI_search_type failed_count suppressed_count runtime_headroom_pct avg_runtime max_runtime run_count
            | sort +runtime_headroom_pct

Hope it helps. Cheers!!!

View solution in original post

jaime_ramirez
Communicator

Hi

There is this lookup included in ITSI that includes all info for each KPI defined:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name

If you want it separated:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name kpis.* as kpis_*
| eval kpis_info = mvzip(mvzip(mvzip(kpis__key, kpis_base_search, "==@@=="), kpis_search_type, "==@@=="), kpis_title, "==@@==")
| fields service_name service_id kpis_info
| mvexpand kpis_info
| eval kpis_info=split(kpis_info, "==@@==")
| eval kpis__key=mvindex(kpis_info, 0)
| eval kpis_base_search=mvindex(kpis_info, 1)
| eval kpis_search_type=mvindex(kpis_info, 2)
| eval kpis_title=mvindex(kpis_info, 3)
| fields - kpis_info

You could also try searching the splunk scheduler log or a version of the KPI Performance Panel included in the ITSI Healthcheck Dashboard:

index=_internal sourcetype=scheduler savedsearch_name="Indicator*"
            | stats count as run_count, count(eval(status!="success")) as failed_count, count(eval(suppressed!="0")) as suppressed_count,
            avg(run_time) as avg_runtime, max(run_time) as max_runtime, earliest(_time) as first, latest(_time) as last
            by savedsearch_name
            | eval KPI_search_type=if(savedsearch_name like "%Shared%", "base", "ad hoc")
            | eval runtime_headroom_pct=round((100-(max_runtime/((last-first)/(run_count-1))*100)),1)
            | eval avg_runtime=round(avg_runtime, 2)
            | eval max_runtime=round(max_runtime, 2)
            | table savedsearch_name KPI_search_type failed_count suppressed_count runtime_headroom_pct avg_runtime max_runtime run_count
            | sort +runtime_headroom_pct

Hope it helps. Cheers!!!

Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...