Splunk ITSI

If an ITSI KPI scheduled search is skipped/continued how do I know which KPI / Service is impacted?

rphillips_splk
Splunk Employee
Splunk Employee

Seeing skipped searches in scheduler.log is a common problem faced especially if you've scheduled too many concurrent searches or in ITSI land too many KPIs. Is there an easy way to map service id, kpi id , service name and saved search name ? When you create a KPI in ITSI a scheduled search is created with a name like "Indicator - Shared - 5ad7aa9759479309e84b84a2 - ITSI Search".

scheduler.log
04-20-2018 03:52:01.745 +0000 INFO SavedSplunker - savedsearch_id="nobody;itsi;Indicator - Shared - 5ad7aa9759479309e84b84a2 - ITSI Search", search_type="scheduled", user="nobody", app="itsi", savedsearch_name="Indicator - Shared - 5ad7aa9759479309e84b84a2 - ITSI Search", priority=default, status=success, digest_mode=1, scheduled_time=1524196320, window_time=0, dispatch_time=1524196320, run_time=0.946, result_count=2, alert_actions="indicator", sid="scheduler_nobodyitsi_RMD5e3cb0aa7dd5f97bc_at_1524196320_9870", suppressed=0, thread_id="AlertNotifierWorker-0"

If these KPI scheduled searches start skipping , my itsi_summary index will be missing data. How do I know which kpi name & service is impacted?

0 Karma
1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

this search (tested in ITSI 3.0.0):
| inputlookup service_kpi_lookup | rename _key as serviceid title as service_name | eval kpi_info = mvzip('kpis._key', 'kpis.title', "==@@==") | fields kpi_info service_name serviceid | mvexpand kpi_info | rex field=kpi_info "(?<kpiid>.+)==@@==(?<kpi_name>.+)" | fields - kpi_info | join kpiid [search index=itsi_summary | dedup search_name |rename search_name as savedsearch_name] | table service_name serviceid kpi_name kpiid savedsearch_name

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

this search (tested in ITSI 3.0.0):
| inputlookup service_kpi_lookup | rename _key as serviceid title as service_name | eval kpi_info = mvzip('kpis._key', 'kpis.title', "==@@==") | fields kpi_info service_name serviceid | mvexpand kpi_info | rex field=kpi_info "(?<kpiid>.+)==@@==(?<kpi_name>.+)" | fields - kpi_info | join kpiid [search index=itsi_summary | dedup search_name |rename search_name as savedsearch_name] | table service_name serviceid kpi_name kpiid savedsearch_name

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...