We are building a new Splunk environment. As we were doing this I noticed that the Windows TA no longer includes a default/indexes.conf file and all the inputs don't specify an index thus all events would go to the main index <yuck>. This kicked off the discussion about what is the best way to handle indexes going forward. Should I create a local/indexes.conf file for each app that does not have one or should I create our own app with an indexes.conf file and just make sure it has the highest precedence so that it would override the indexes.conf file that may come bundled with any app? I can see that having an indexes.conf file in each app makes it easy to see what app that data goes with. But having our own app for handling all the index makes it easier to make adjustments to all the indexes without having to edit multiple files.
FYI, we have clustered indexers so I cannot just rely on the web UI for this.
In a clustered environment you should have one indexes.conf that is centrally located on your cluster master (CM). Any changes made to the indexes.conf should be done on the CM, followed by a bundle push to propagate the indexes.conf file to your clustered search peers (indexers).
What I believe you're talking about is your inputs.conf, which will have all of the files/dirs you want to monitor and send to your indexers. The Windows TA by default doesn't have any indexes listed in the supplied inputs.conf, so these settings will have to be set by you.
Never edit anything in a default directory. What you want to do is copy or create an inputs.conf file in ../Splunk_TA_windows/local and add your indexes entries. For example...
[WinEventLog://Security] disabled = 0 index = windows
Hope that helped!
I know that you never edit the default conf files and that in a clustered environment you put the apps in the master-apps folder on the cluster master and then push to the indexers. That wasn't really my question.
Many apps come with their own default/indexes.conf file. The Windows TA even did until recently-ish. So, my question is often times we need to override what is in the default/indexes.conf file. Is it best to create a app/local/indexes.conf file or create our own app and handle all indexes through our own app. If it is best to create our own app what have other found is the best way to have that app take precedence. My understanding is that precedence is determined alphabetically. Yes, I know there is more to it than that but if we create our own app lexicographical order would be the part that would come in to play. Should I create an app called Aaa_My_Config_App or something like that? Or is it best to keep the indexes with the app they are for in the app/local/indexes.conf file so that you don't have to deal with file precedence and you know what app the indexes are for?