I don't think you will see full documentation, but can say it is a bad practice to deploy all apps on all Splunk components
Risk with inputs.conf => Same inputs.conf will cause data duplication and unwanted data coming. (eg. You had put to collect from /var/log/* on certain system. But this will be damaging on other systems as some application will put /var/log/myapplication/* and can fill your index within seconds)
Risk with props.conf => Sourcetypes will be messed up if you don't properly align data. (eg. you monitor /var/log/syslog/app1 with sourcetype app1_sourcetype. But if there is another inputs with /var/log/syslog/*, you don't know how the sourcetype will be assigned)
complexity with merged conf files=> if you run a btool , you will be surprised to see how each stanza would get preference. Basically, you will never expect to see things as per your code. Without naming convention and config file precedence control, you will be entering into wild wild west for enterprise systems.
Risk of data going to wrong index => Of course, you don't have control where data goes as some default props/inputs will send to default indexes of the TA.
Search Head Clusters => All the best !! With user inputs coming to local directory and your configs pushed to default, better to leave the company than to administer it 🙂
In my opinion, without best practices, control of configs and correct naming conventions it is very hard to administer in large enterprise systems.
... View more