I have a Splunk Enterprise Search Head in a Production and a second one in a Non-Prod environment. Any best practices fo

[adnankhan5133]

The search head in the Non-Prod environment will not be active and would only be turned on in the event of a disaster where the Production SH is down.

I was thinking about enabling an rsync between both search heads so that the conf. files and knowledge objects from the Prod SH are regularly synced over to the Non-Prod SH. Does anyone have any suggestions or better approaches?

[esix_splunk · Accepted solution]

Rsync will work fine for this. Be cautious around the GUIDs of the SH, if you bring production back up while the DR is running you can have some potential issues. You could change this easily enough and not sync this config file.

If you're syncing KOs, why no use GIT or similar repo control. This is what most companies are doing now these days. It's a lot easier for granular control of what you replicate across, not to mention the benefits of version control/tracking. 

[adnankhan5133]

If the Production SH went down, how would Git sync the changes over to the Non-Prod/Secondary SH? If there is an article or an app that gracefully syncs all knowledge objects between search heads, then that would be ideal for me to check out.

Sorry, I'm new to Git and came from a world where rsync was the answer to replicating KO's between search heads for DR purposes.

[adnankhan5133]

Agreed - Git or Ansible is definitely the way to go. I consulted with several others and that appears to be the best path forward.