How to get list of summary index and sourcetype in Splunk

[harishsplunk7]

I want to get the list of summary index configured in splunk. Please help me with queries to get the summary index and sourcetype

[m_pham]

Any non-internal indexes could be a summary index to be honest. But like @dtburrows3 said, you'll have to take a look at savedsearches.conf to see what search is using the collect command that writes to an index. This isn't guaranteed to identify summary indexes but will help you narrow down what indexes to look into. In our environment, our summary indexes are identified with the "summary_" prefix as best practice.

[dtburrows3]

As far as I know, any index that receives results of a scheduled report is considered a summary index (i.e. using the collect command in a search or configuration of the "action.summary_index" parameter in savedsearches.conf.

To look for savedsearches using either one of these methods you can search the rest endpoint like this.

| rest splunk_server=local /servicesNS/-/-/saved/searches
    | fields + title, qualifiedSearch, "action.summary_index", "action.summary_index.*"
    | where match(qualifiedSearch, "(?i)\|(?:\s|\n)*collect") OR ('action.summary_index'=="1" OR match('action.summary_index', "(?i)true"))
    | rename
        title as savedsearch_name
    | rex field=qualifiedSearch max_match=0 "(?<collect_spl>\|\s*collect\s+[^\n]+)"
    | fields + savedsearch_name, collect_spl, "action.summary_index", "action.summary_index.*"

From here you could set up regex to extract index/sourcetype from the "collect_spl" field or use the "action.summary_index.*" values to gather that info.

Its possible for the "collect_spl" field to contain only index and even then, that index specification could be stored in a macro, so those situations may be a bit more tricky.

It is also possible for a parameter called "output_format=hec" to be used along with the collect command and if this is the case then, sourcetype and source will not be specified with the collect command and are rather defined in the SPL itself.

You can see examples of these scenarios here

To use this method to the end result of a report listing index/sourcetypes that are being utilized as a summary index you can use SPL like this. (Note: there is a custom splunk command being used in this code that expands macros all the way down before we attempt to do any extractions of collect metadata. You can DM me if you would want me to share the script to do this)

| rest splunk_server=local /servicesNS/-/-/saved/searches
    | fields + title, qualifiedSearch, "action.summary_index", "action.summary_index.*"
    | where match(qualifiedSearch, "(?i)\|(?:\s|\n)*collect") OR ('action.summary_index'=="1" OR match('action.summary_index', "(?i)true"))
    | rename
        title as savedsearch_name
    ``` this is a splunk custom command I created, reach out to me through DM and I can share the code ```
    | expandmacros input_field=qualifiedSearch output_field=expanded_spl
    | rex field=expanded_spl max_match=0 "(?<collect_spl>\|\s*collect\s+[^\n]+)"
    | where isnotnull(collect_spl) OR ('action.summary_index'=="1" OR match('action.summary_index', "(?i)true"))
    | fields + savedsearch_name, collect_spl, expanded_spl, "action.summary_index", "action.summary_index.*"
    | rex field=expanded_spl max_match=0 "(?i)\|\s*(?<eval_spl>eval\s+[^\|]+)"
    | eval
        eval_spl=mvfilter(match(eval_spl, "\s+source(?:type)?\"?\s*\=\s*\""))
    | rex field=eval_spl max_match=0 "\s+sourcetype\"?\s*\=\s*\"(?<inline_set_sourcetype>[^\"]+)"
    | rex field=eval_spl max_match=0 "\s+source\"?\s*\=\s*\"(?<inline_set_source>[^\"]+)"
    | rex field=collect_spl max_match=0 "index\s*\=\s*\"?(?<summary_index>[a-zA-Z0-9\-\_]+)"
    | rex field=collect_spl max_match=0 "sourcetype\s*\=\s*\"?(?<summary_sourcetype>[a-zA-Z0-9\-\_]+)"
    | rex field=collect_spl max_match=0 "source\s*\=\s*\"?(?<summary_source>[a-zA-Z0-9\-\_]+)"
    | fields + savedsearch_name, collect_spl, summary_index, summary_sourcetype, summary_source, inline_set_sourcetype, inline_set_source, "action.summary_index", "action.summary_index.*"
    | eval
        summary_index=mvdedup(
            mvappend(
                'summary_index',
                'action.summary_index._name'
                )
            ),
        summary_sourcetype=mvdedup(
            mvappend(
                summary_sourcetype,
                inline_set_sourcetype
                )
            ),
        summary_source=mvdedup(
            mvappend(
                summary_source,
                inline_set_source
                )
            )
    | fillnull value="stash" summary_sourcetype
    | fields - inline_*
    | stats
        dc(savedsearch_name) as dc_savedsearches
            by summary_index, summary_sourcetype
    | sort 0 -dc_savedsearches

 Final output would look something like this. (screenshot has been redacted)