Remote Site local searches when connection to main site is lost

[michaelhc]

Hey guys, I'm pretty new to Splunk but I've been reading the documentation and trying to understand how I could implement my desired setup.

I have several remote sites, which will have a number of UFs at each of them. Additionally, I have one main site from which I want to conduct my searches from. However, I would also like to have the capability for each remote site to search only through its local data. This would be in the case that connectivity to the main site is lost, so that each remote site can function as its own mini setup. Furthermore, if one of the remote sites goes down I still need to be able to perform searches for the indexed data from my main site. For access control reasons I need to ensure that searches made from a remote site can only search data from the same remote site.

My original idea was to have several single site indexer clusters each with their own search head, and additionally have a search head at my main site which searches across all of the single site indexer clusters. However, I don't believe this would allow me to continue to conduct searches on indexed data for a remote site from the main site if the remote site were to go down. So I somehow need to replicate the indexed data only from the remote sites to the main site, without replicating from main to remote or remote to remote. I think in theory I could do this by creating an equal number of multi site indexer clusters to my number of remote sites, with the second site being the main site in each, and searching across these, but I feel like there must be a more efficient way of achieving the functionality that I'm looking for.

Any suggestions?

Thanks for any help,

Mike.

[woodcock · Accepted solution]

Create a multi-site indexer cluster and a main search head that is peered to the cluster. Then create another search head in each site that is also peered to the cluster but with site-affinity set to prefer the local site. Now all you have to do is a daily rsync from the main search head to copy $SPLUNK_HOME/etc/* from the main search head to the site-local emergency search heads.

[michaelhc]

Thanks for the response.

I had a look at search affinity, and I noticed that it said "For those sites that you want to support search affinity, you must configure multisite clustering so that the site has a full set of searchable data and a local search head."

By full set I am assuming this means that a user from remote site A would be able to search data collected from forwarders at remote site B, but that data would be stored at both sites A and B, and the search head at site A would only search through the data stored at site A.

Would it be possible for site A to only store data from site A UFs, and site B only stores data from site B UFs? I would like the remote sites to be completely separate from one another, and the only location where one could view data from any site should be the main site.

Thanks so much.

[woodcock]

No, the feature does not work that way. Each site contains ALL of the data from EVERY site.

[gcusello]

HI michaelhc,
you need a multisite Indexers Cluster ( https://docs.splunk.com/Documentation/Splunk/7.3.2/Indexer/Multisitedeploymentoverview ).
But it isn't an architecture that you can create without an expert and certified Splunk Architect.

Ciao.
Giuseppe