How to get list of summary index and sourcetype in Splunk

harishsplunk7 — Tue, 02 Jan 2024 18:01:05 GMT

I want to get the list of summary index configured in splunk. Please help me with queries to get the summary index and sourcetype

Re: How to get list of summary index and sourcetype in Splunk

dtburrows3 — Tue, 02 Jan 2024 19:28:06 GMT

As far as I know, any index that receives results of a scheduled report is considered a summary index (i.e. using the collect command in a search or configuration of the "action.summary_index" parameter in savedsearches.conf.

To look for savedsearches using either one of these methods you can search the rest endpoint like this.

From here you could set up regex to extract index/sourcetype from the "collect_spl" field or use the "action.summary_index.*" values to gather that info.

Its possible for the "collect_spl" field to contain only index and even then, that index specification could be stored in a macro, so those situations may be a bit more tricky.

It is also possible for a parameter called "output_format=hec" to be used along with the collect command and if this is the case then, sourcetype and source will not be specified with the collect command and are rather defined in the SPL itself.

You can see examples of these scenarios here

To use this method to the end result of a report listing index/sourcetypes that are being utilized as a summary index you can use SPL like this. (Note: there is a custom splunk command being used in this code that expands macros all the way down before we attempt to do any extractions of collect metadata. You can DM me if you would want me to share the script to do this)

| rest splunk_server=local /servicesNS/-/-/saved/searches | fields + title, qualifiedSearch, "action.summary_index", "action.summary_index.*" | where match(qualifiedSearch, "(?i)\|(?:\s|\n)*collect") OR ('action.summary_index'=="1" OR match('action.summary_index', "(?i)true")) | rename title as savedsearch_name ``` this is a splunk custom command I created, reach out to me through DM and I can share the code ``` | expandmacros input_field=qualifiedSearch output_field=expanded_spl | rex field=expanded_spl max_match=0 "(?<collect_spl>\|\s*collect\s+[^\n]+)" | where isnotnull(collect_spl) OR ('action.summary_index'=="1" OR match('action.summary_index', "(?i)true")) | fields + savedsearch_name, collect_spl, expanded_spl, "action.summary_index", "action.summary_index.*" | rex field=expanded_spl max_match=0 "(?i)\|\s*(?<eval_spl>eval\s+[^\|]+)" | eval eval_spl=mvfilter(match(eval_spl, "\s+source(?:type)?\"?\s*\=\s*\"")) | rex field=eval_spl max_match=0 "\s+sourcetype\"?\s*\=\s*\"(?<inline_set_sourcetype>[^\"]+)" | rex field=eval_spl max_match=0 "\s+source\"?\s*\=\s*\"(?<inline_set_source>[^\"]+)" | rex field=collect_spl max_match=0 "index\s*\=\s*\"?(?<summary_index>[a-zA-Z0-9\-\_]+)" | rex field=collect_spl max_match=0 "sourcetype\s*\=\s*\"?(?<summary_sourcetype>[a-zA-Z0-9\-\_]+)" | rex field=collect_spl max_match=0 "source\s*\=\s*\"?(?<summary_source>[a-zA-Z0-9\-\_]+)" | fields + savedsearch_name, collect_spl, summary_index, summary_sourcetype, summary_source, inline_set_sourcetype, inline_set_source, "action.summary_index", "action.summary_index.*" | eval summary_index=mvdedup( mvappend( 'summary_index', 'action.summary_index._name' ) ), summary_sourcetype=mvdedup( mvappend( summary_sourcetype, inline_set_sourcetype ) ), summary_source=mvdedup( mvappend( summary_source, inline_set_source ) ) | fillnull value="stash" summary_sourcetype | fields - inline_* | stats dc(savedsearch_name) as dc_savedsearches by summary_index, summary_sourcetype | sort 0 -dc_savedsearches

Final output would look something like this. (screenshot has been redacted)

Re: How to get list of summary index and sourcetype in Splunk

m_pham — Tue, 02 Jan 2024 21:19:40 GMT

Any non-internal indexes could be a summary index to be honest. But like @dtburrows3 said, you'll have to take a look at savedsearches.conf to see what search is using the collect command that writes to an index. This isn't guaranteed to identify summary indexes but will help you narrow down what indexes to look into. In our environment, our summary indexes are identified with the "summary_" prefix as best practice.

topic Re: How to get list of summary index and sourcetype in Splunk in Getting Data In

How to get list of summary index and sourcetype in Splunk

Re: How to get list of summary index and sourcetype in Splunk

Re: How to get list of summary index and sourcetype in Splunk