Getting Data In

How to use indexer master as listener for data, but actual data will be stored on slaves?

Rialf1959
Explorer

Hello,
How to use indexer master as listener for data, but actual data will be stored on slaves?
Is possible to forward data from universal forwarders to indexer master and have random number of indexer slaves?
With this config - data are stored in indexer master. Where is a problem? Thanks

version: '3.5'
services:
  master:
    hostname: master
    image: splunk/splunk:7.0.2
    ports:
      - 8001:8000
      - 9997:9997
    environment:
     - "SPLUNK_ENABLE_LISTEN=9997"
     - "SPLUNK_START_ARGS=--accept-license --answer-yes "
     - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
     - "SPLUNK_CMD_2=edit cluster-config -mode master -replication_factor 1 -search_factor 1 -secret mytoken -cluster_label mycluster"
     - "SPLUNK_CMD_3=set servername splunk-master.example.com"
     - "SPLUNK_CMD_4=set default-hostname splunk-master.example.com"
     - "SPLUNK_CMD_5=restart"


  head:
    hostname: head
    image: splunk/splunk:7.0.2
    ports:
      - 8002:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode searchhead -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-head.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-head.example.com"
      - "SPLUNK_CMD_5=restart"

  slave:
    hostname: slave
    image: splunk/splunk:7.0.2
    ports:
      - 8003:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode slave -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-slave.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-slave.example.com"
      - "SPLUNK_CMD_5=restart"
Tags (2)
0 Karma

tiagofbmm
Influencer

Are you forwarding the data from the Cluster Master to the Indexers?

In the cluster master

1 - Create an outputs.conf pointing to the indexer:

https://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Configureforwardingwithoutputs.conf

2 - Create an app in the $SPLUNK_HOME/etc/master-apps and put an inputs.conf there with the stanza:

[splunktcp:<port_you_are_sending_from_the_Cluster_master>]

Then in the Cluster Master

$SPLUNK_HOME/bin/splunk validate cluster-bundle
$SPLUNK_HOME/bin/splunk apply cluster-bundle 

And with this you will be forwarding all the data arriving to the Cluster Master to the indexer layer like you want

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This makes the Master a choke point and SPOF. Not a good idea, IMO.

---
If this reply helps you, Karma would be appreciated.
0 Karma

tiagofbmm
Influencer

Agreed, not a good idea indeed. But if for some reason the user is into using the CM as a Listener, I believe this would be the way

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Master Node is not a data forwarder. That is, you can't send data to it and expect that data to land on a search peer.

It looks like you want Indexer Discovery. This feature tells the forwarders to ask the Master Node for a list of indexers. The forwarders then use that list to determine where to send data.
See http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/indexerdiscovery for details.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Rialf1959
Explorer

This is not working in Docker because if a container is not running with --network host,
master node will forward internal ip address, not public one -> is not accessible.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...