Dashboards & Visualizations

How do we sync user created dashboards and saved searches between 2 search heads that are accessed using a load balanced URL?

shahzadarif
Path Finder

We've got 2 search heads in our environment and users access them using a load balanced URL. That's all working as expected, however, we've got an issue and I just don't know how to resolve it.

Users are creating dashboards and saving searches which obviously are staying on the machine they're currently logged in. Once they log out and back in, if the load balancer takes them to the second search head, they don't see their saved searches or dashboards. I fixed this issue by creating a cron job which rsyncs apps directory once a day, but obviously this isn't as frequent as they want. I had it set to run every 2 minutes but that was causing issues like if a user deleted a saved search it would get sync'd.

Is there another way to resolve this issue besides rsync? Can I mount apps directory on a shared NFS storage or something?

0 Karma

jplumsdaine22
Influencer

What you want is search head clustering. It will replicate knowledge objects between search heads. Unfortunately you're going to need another search head, as you need at least three in a cluster.

Check out the docs on search head clustering: http://docs.splunk.com/Documentation/Splunk/6.3.1/DistSearch/AboutSHC

0 Karma

shahzadarif
Path Finder

Thanks for getting back to me so quickly.
I've got Cluster Manager, can I use that as a 3rd Search Head? Thanks

0 Karma

jplumsdaine22
Influencer

All the answers are in the documentation. It's fairly involved so I recommend having a good read through it. It's not as simple as adding A records to DNS as you would for simple web servers, as there is a great deal of state replication (not to mention the distribution of scheduled searches between cluster members)

The benefit of doing this work is great though. You get reliable failover and replication, and it doesn't cost you a cent extra. In my experience, adding members to a cluster seems to be fairly easy once the cluster is set up, so you can scale out easily in the future.

Also I strongly recommend putting a load balancer in front of your cluster instead of a round robin dns entry. Check out HAproxy if you don't have one already.

0 Karma
Get Updates on the Splunk Community!

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...