Splunk IT Service Intelligence

Howto get a list of ITSI services with source of KPI base search

krusovice
Path Finder

Hello,

In my environment, I have a long list of ITSI services (created by someone else) which using default KPI base search. These default KPI base search is running every mins for 1 min data and it has causes some impact to the indexers.

Without going through the UI for ITSI services and checking the search source for each KPI, is there any query I can use to generate the list of the ITSI services + KPI base search + base search source name?

Thanks.

Tags (1)
1 Solution

jaime_ramirez
Communicator

Hi

There is this lookup included in ITSI that includes all info for each KPI defined:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name

If you want it separated:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name kpis.* as kpis_*
| eval kpis_info = mvzip(mvzip(mvzip(kpis__key, kpis_base_search, "==@@=="), kpis_search_type, "==@@=="), kpis_title, "==@@==")
| fields service_name service_id kpis_info
| mvexpand kpis_info
| eval kpis_info=split(kpis_info, "==@@==")
| eval kpis__key=mvindex(kpis_info, 0)
| eval kpis_base_search=mvindex(kpis_info, 1)
| eval kpis_search_type=mvindex(kpis_info, 2)
| eval kpis_title=mvindex(kpis_info, 3)
| fields - kpis_info

You could also try searching the splunk scheduler log or a version of the KPI Performance Panel included in the ITSI Healthcheck Dashboard:

index=_internal sourcetype=scheduler savedsearch_name="Indicator*"
            | stats count as run_count, count(eval(status!="success")) as failed_count, count(eval(suppressed!="0")) as suppressed_count,
            avg(run_time) as avg_runtime, max(run_time) as max_runtime, earliest(_time) as first, latest(_time) as last
            by savedsearch_name
            | eval KPI_search_type=if(savedsearch_name like "%Shared%", "base", "ad hoc")
            | eval runtime_headroom_pct=round((100-(max_runtime/((last-first)/(run_count-1))*100)),1)
            | eval avg_runtime=round(avg_runtime, 2)
            | eval max_runtime=round(max_runtime, 2)
            | table savedsearch_name KPI_search_type failed_count suppressed_count runtime_headroom_pct avg_runtime max_runtime run_count
            | sort +runtime_headroom_pct

Hope it helps. Cheers!!!

View solution in original post

a1bg503461
Explorer

I want to download the service_kpi_sbs_lookup.csv file and import it in another test splunk whit Lookup Editor. Does anyone know where is service_kpi_sbs_lookup.csv under linux ? Thanks

0 Karma

jaime_ramirez
Communicator

Hi

There is this lookup included in ITSI that includes all info for each KPI defined:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name

If you want it separated:

| inputlookup service_kpi_sbs_lookup
| rename _key as service_id title as service_name kpis.* as kpis_*
| eval kpis_info = mvzip(mvzip(mvzip(kpis__key, kpis_base_search, "==@@=="), kpis_search_type, "==@@=="), kpis_title, "==@@==")
| fields service_name service_id kpis_info
| mvexpand kpis_info
| eval kpis_info=split(kpis_info, "==@@==")
| eval kpis__key=mvindex(kpis_info, 0)
| eval kpis_base_search=mvindex(kpis_info, 1)
| eval kpis_search_type=mvindex(kpis_info, 2)
| eval kpis_title=mvindex(kpis_info, 3)
| fields - kpis_info

You could also try searching the splunk scheduler log or a version of the KPI Performance Panel included in the ITSI Healthcheck Dashboard:

index=_internal sourcetype=scheduler savedsearch_name="Indicator*"
            | stats count as run_count, count(eval(status!="success")) as failed_count, count(eval(suppressed!="0")) as suppressed_count,
            avg(run_time) as avg_runtime, max(run_time) as max_runtime, earliest(_time) as first, latest(_time) as last
            by savedsearch_name
            | eval KPI_search_type=if(savedsearch_name like "%Shared%", "base", "ad hoc")
            | eval runtime_headroom_pct=round((100-(max_runtime/((last-first)/(run_count-1))*100)),1)
            | eval avg_runtime=round(avg_runtime, 2)
            | eval max_runtime=round(max_runtime, 2)
            | table savedsearch_name KPI_search_type failed_count suppressed_count runtime_headroom_pct avg_runtime max_runtime run_count
            | sort +runtime_headroom_pct

Hope it helps. Cheers!!!

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...