Deployment Architecture

Can I configure apps to only use specific index clusters when search heads are connected to multiple clusters?

ngablern
New Member

So I am in somewhat of a fun situation where we have multiple instances of Splunk installed each with their own index clusters and search head clusters.  I know you can configure search heads to search multiple index clusters, but not all my index clusters have the "same" data in a named index (mainly, index=main).  So what I was wondering is if I install all the apps from all of the instances onto the search head cluster that is configured to connect to all index clusters, I can tell those apps to only look to the appropriate index cluster that has the data they want?  I think I could accomplish this with sites maybe if I can tie an app to a site.  But I am not finding either index cluster or site configurations for individual apps.  The point would be to provide a single place to login and be able to see all the splunk data and to eventually retire the now extraneous search head clusters without the apps having to search multiple "main" indexes in clusters that don't have the data they are looking for.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

While there is no feature to really easily do this, you can use search groups and macros to do this...

These do require use training and understanding when the craft their searches..

 

Above being said, in large scale deployment planning there are some points you can take into account for.. 

Do note that in the latest versions of Splunk, we don't distribute searches to clusters where the index names dont exist (the cm will report this back in SH.) So searching index=security* would be similar to searching index=security* splunk_server_group=securitycluster.

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There's no such feature and I hope there never is.  While I can see the benefit, I would not want to be the admin that has to keep track of what data is on each cluster so I can point newly-installed apps to the right place(s).

And how can we be sure app A that's currently just accessing data from cluster 1 can't benefit from data in cluster 2 or 3?

The only way I can think of to get the performance improvement you're thinking of is to have a different set of indexes on each cluster.  Then, when an app needs to access data in a certain index the request will only go to the one cluster that has it.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

gjanders
SplunkTrust
SplunkTrust

This idea https://ideas.splunk.com/ideas/EID-I-103 exists to solve this. Please up vote if interested 

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.