Getting Data In

How to use indexer master as listener for data, but actual data will be stored on slaves?

Rialf1959
Explorer

Hello,
How to use indexer master as listener for data, but actual data will be stored on slaves?
Is possible to forward data from universal forwarders to indexer master and have random number of indexer slaves?
With this config - data are stored in indexer master. Where is a problem? Thanks

version: '3.5'
services:
  master:
    hostname: master
    image: splunk/splunk:7.0.2
    ports:
      - 8001:8000
      - 9997:9997
    environment:
     - "SPLUNK_ENABLE_LISTEN=9997"
     - "SPLUNK_START_ARGS=--accept-license --answer-yes "
     - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
     - "SPLUNK_CMD_2=edit cluster-config -mode master -replication_factor 1 -search_factor 1 -secret mytoken -cluster_label mycluster"
     - "SPLUNK_CMD_3=set servername splunk-master.example.com"
     - "SPLUNK_CMD_4=set default-hostname splunk-master.example.com"
     - "SPLUNK_CMD_5=restart"


  head:
    hostname: head
    image: splunk/splunk:7.0.2
    ports:
      - 8002:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode searchhead -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-head.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-head.example.com"
      - "SPLUNK_CMD_5=restart"

  slave:
    hostname: slave
    image: splunk/splunk:7.0.2
    ports:
      - 8003:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode slave -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-slave.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-slave.example.com"
      - "SPLUNK_CMD_5=restart"
Tags (2)
0 Karma

tiagofbmm
Influencer

Are you forwarding the data from the Cluster Master to the Indexers?

In the cluster master

1 - Create an outputs.conf pointing to the indexer:

https://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Configureforwardingwithoutputs.conf

2 - Create an app in the $SPLUNK_HOME/etc/master-apps and put an inputs.conf there with the stanza:

[splunktcp:<port_you_are_sending_from_the_Cluster_master>]

Then in the Cluster Master

$SPLUNK_HOME/bin/splunk validate cluster-bundle
$SPLUNK_HOME/bin/splunk apply cluster-bundle 

And with this you will be forwarding all the data arriving to the Cluster Master to the indexer layer like you want

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This makes the Master a choke point and SPOF. Not a good idea, IMO.

---
If this reply helps you, Karma would be appreciated.
0 Karma

tiagofbmm
Influencer

Agreed, not a good idea indeed. But if for some reason the user is into using the CM as a Listener, I believe this would be the way

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Master Node is not a data forwarder. That is, you can't send data to it and expect that data to land on a search peer.

It looks like you want Indexer Discovery. This feature tells the forwarders to ask the Master Node for a list of indexers. The forwarders then use that list to determine where to send data.
See http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/indexerdiscovery for details.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Rialf1959
Explorer

This is not working in Docker because if a container is not running with --network host,
master node will forward internal ip address, not public one -> is not accessible.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...