We're building a Splunk app and need to decide which index to use to store the events captured by our app's inputs. We'll let the use choose the index in our app's Setup configuration page (aka setup.xml), but we need to decide on a default.
Got a suggestion for whether our default should be "main" or should be a custom index for the app? If "it depends" then when would we want to choose one option over the other?
I would generally recommend that data go to the system
main) index by default. There is a significant environment-specific and system-level configuration for defining an index (disk space, volumes, retention policy, etc), and by specifying a new one, you would be significantly complicating the (correct) installation of your application. Separate identification of your data can be accomplished with distinct sourcetypes.
A separate index is important if you need to be able to move, delete, age, archive (etc) this data differently from the rest of the data in Splunk. You can and should think about doing this, but I would say that you should think about it separately from installation and configuration of the app. If necessary, it should be done before app installation. You can define these parameters for the index, and then tell your app inputs to use that index. But you should not force this by default.