I have two separate Splunk environments: 1) syslog data for platform group 2) network data for LAN & WAN
I don't want Env #1 doing a distributed search to #2 unless we are troubleshooting a specific outage. How can I easily turn on/off distributed searches between separate Splunk environments? Would it be as simple as adding the Indexing server #2 when troubleshooting and removing when done? Or is there a better method to have this capability? And if we have a third environment or fourth environment.....could it easily expand to search them all during troubleshooting times only?
Might be a "feature" for upcoming Splunk versions to offer options to turn "on/off" cross environment searching.
Master Oogway
Seems there is a solution in version 6.2
http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Distributedsearchgroups
If nothing is defined (standard), all distributed peers will be searched. However, you'd be able to that with users & roles. Use different users, e.g. standard-user for "normal" use (in this case you have to limitate user/roles priveleges for "normal" user) and like a "debug-user" to troubleshoot (no limitation).
Have a look in these docus