Monitoring Splunk
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Unable to locate an inputs stanza sending a data to a particular index using btool ?

Hemnaath
Motivator

Hi All, we are removing the unwanted indexes from prod environment as they are deprecated, while performing this activity, we got stuck in finding the exact inputs.conf stanza monitoring and sending the data to a particular index, I had used below steps to find out the inputs stanza to disable and remove it from the host, but unable to get the index details in inputs.conf stanza.

steps:
1) From search head, we have got the host, source and sourcetype details by searching index=summary_forwarders.
2) Next logged into the particular indexer host from where the data are being sent to the index=summary_forwardes and executed the btool command "./splunk cmd btool --debug inputs list | grep summary_forwarders | more but unable to locate the stanza.
3) Tired to access the indexer instance via GUI but unable to access the web URL.

Kindly let me know is there any other method where we can filter out index detail from inputs.conf stanza details.

Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

An index name like "summary_forwarders" sounds like a summary index. If that's what it is then you won't (shouldn't) find any references to it in inputs.conf. Summary indexes are written to by scheduled searches. Look in your savedsearches.conf files for references to that index.

find $SPLUNK_HOME -name savedsearches.conf -print0 | xargs -r0 grep summary_forwarders
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Hemnaath
Motivator

Hi richgalloway, thanks for your effort on this, but when tried the above find command, it did not fetch any output.

Need to find for all the below list of indexes details
summary_forwarders
summary_indexers
summary_sourcetypes
summary_sources
summary_hosts
summary_pools

Kindly guide me, is there any other way i can narrow down the inputs.conf details for the above list of indexes details.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

For each index, search for the sources that write to that index. Then search inputs.conf for those sources.

| metadata type=sources index=summary_forwarders

If you use universal forwarders then you will need to search in the inputs.conf files on the forwarders (unless you have local copies).

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Hemnaath
Motivator

Hi richgalloway, I had tried above search and found the source type and source, then ran the btool to find out the location of the inputs.conf stanza in the indexer instance host but no luck.

index=summary_forwarders source="All forwarders - regenerator summary index" sourcetype=stash

btool details :
./splunk cmd btool inputs list --debug | grep All forwarders | more
grep: forwarders: No such file or directory
so decided to run the btool against the sourcetype=stash and found some output but there was no source mapped to it.
btool details:
./splunk cmd btool inputs list --debug | grep stash | more
/opt/splunk/etc/system/default/inputs.conf

[batch:///opt/splunk/var/spool/splunk/...stash_new]
/opt/splunk/etc/system/default/inputs.conf queue = stashparsing
/opt/splunk/etc/system/default/inputs.conf sourcetype = stash_new

Executed the btool against the props.conf file to find out the location, but no luck.
./splunk cmd btool props list --debug | grep stash | more
/opt/splunk/etc/system/default/props.conf [source::...stash]
/opt/splunk/etc/system/default/props.conf sourcetype = stash
/opt/splunk/etc/system/default/props.conf [source::...stash_new]
/opt/splunk/etc/system/default/props.conf sourcetype = stash_new
/opt/splunk/etc/system/default/props.conf [stash]
/opt/splunk/etc/system/default/props.conf REPORT-1 = stash_extract
/opt/splunk/etc/system/default/props.conf [stash_new]
/opt/splunk/etc/system/default/props.conf TRANSFORMS-sourcetype = set_sourcetype_to_stash

kindly guide how to get this fixed, we need to remove above mentioned index details from indexes.conf before removing we want to stop these index to ingest data into it.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Like I suspected originally, you're working with summary indexes. This is shown by "sourcetype=stash". That means you will not find references to the index in inputs.conf. Look in savedsearches.conf.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Hemnaath
Motivator

HI richgalloway, you're right, i could see the config details for all the indexes which are listed in the above comments in savedsearches.conf, so by disabling the index details in savedsearches.conf will stop sending the data to these indexes, is that correct. And also please guide us why it is configured like this usually monitoring details will be in inputs.conf right.

Btool details:

./splunk cmd btool savedsearches list --debug | grep summary_forwarders | more
/opt/splunk/etc/apps/SplunkDeploymentMonitor/default/savedsearches.conf search = index="summary_forwarders" | delete
/opt/splunk/etc/apps/SplunkDeploymentMonitor/default/savedsearches.conf action.summary_index._name = summary_forwarders
/opt/splunk/etc/apps/SplunkDeploymentMonitor/default/savedsearches.conf search = index="summary_forwarders" | eval mb=kb/1024 | eval _time = _time+1800 | timechart pa
rtial=f sum(mb) as MB by sourceHost

thanks in advance.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You have a saved search, defined in "action.summary_index._name" line, that puts its results into a summary index called "summary_forwarders". You have a two options.
1) Disable this search so it doesn't run and doesn't write to the index.
2) Change the index name so it writes to a different index.

You also have a couple of searches that are reading data from the 'summary_forwarders' index. These searches must be disabled or modified before the index can be deleted.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!