Deployment Architecture

Can I use indexer discovery to forward search head cluster data to multi-site indexer cluster?

basanthp
Path Finder

We have a setup of 6 search head clusters connected to 2 multi-site indexer clusters, using Splunk v6.3.1. My objective is to forward search head cluster data to multi-site indexer cluster.

As per following documentation, I understand this can be achieved using outputs.conf
http://docs.splunk.com/Documentation/Splunk/6.3.1/DistSearch/Forwardsearchheaddata
tcpoutput stanza (server=host1, host2, host3)

But I want the cluster master node to handle this (similar to indexer discovery in forwarders).

q1. Is this achievable?
q2. Will the data be replicated if it is sent to a set of indexers in multi-site indexers?

0 Karma
1 Solution

basanthp
Path Finder

Yes. Technically possible. We need to add the role of forwarder to search head. We also need to disable indexing on the SH so data is not duplicated according to the following section.

“2. Configure the search head as a forwarder. Create an outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.”

Since the SH now is also a forwarder, we should be able to use the outputs.conf like, on any other forwarder, so indexerDiscovery should work the same. Forwarder contacts the cluster master and gets the list of indexers.

We have tested this in our LAB and implemented in PROD.

View solution in original post

0 Karma

basanthp
Path Finder

Yes. Technically possible. We need to add the role of forwarder to search head. We also need to disable indexing on the SH so data is not duplicated according to the following section.

“2. Configure the search head as a forwarder. Create an outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.”

Since the SH now is also a forwarder, we should be able to use the outputs.conf like, on any other forwarder, so indexerDiscovery should work the same. Forwarder contacts the cluster master and gets the list of indexers.

We have tested this in our LAB and implemented in PROD.

0 Karma

sylvainlectra
Explorer

Hi,

I've copied my outputs.conf from my other forwarders on my search head in /opt/splunk/etc/system/local/outputs.conf :

[indexer_discovery:master1]
pass4SymmKey = ************
master_uri = https://*********:8089

[tcpout:group1]
autoLBFrequency = 30
forceTimebasedAutoLB = true
indexerDiscovery = master1
useACK = true

[tcpout]
defaultGroup = group1

After a restart I'm able to add monitors to the indexes present on the peers which seems to mean that discovery occured but I see no data from the heads when I do a search and "list forward-server" on the search heads returns nothing

root@********-spk-search1:/opt/splunk/etc# /opt/splunk/bin/splunk list forward-server
Active forwards:
        None
Configured but inactive forwards:
        None

Did I miss something ?

Also how do I disable local indexing on the search heads ?

Regards.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...