topic Re: index replication in custom index in Getting Data In

index replication in custom index

Prakhar_shukla — Tue, 29 Sep 2020 13:59:30 GMT

Hello,
i have created a new index DAP in cluster master and shared the configuration of this new indexes.conf with all peers.
i put the file in cluster master folder ../_cluster/local/ then distribute the bundle to all peer.
i have CM1 , IDX1,IDX2,IDX3 and IDX4.
Now from Heavy forwarder i am forwarding the data to IDX2 index name DAP. In search i am able to search the data,

My question here is - 1) why DAP index is not replicating? all the time in search head, i am getting my data from IDX2 DAP index, why not from other IDXs
2) Can i directly forward heavy forwarder data to cluster master index DAP? will it work?

My conf in HF is -

inputs.conf

[monitor:///tmp/Gov.csv]
disabled = false
index = dap
_TCP_ROUTING = DAP

outputs.conf-

[tcpout:DAP]
server = IDX2:9997
useACK = true
[tcpout-server://IDX2:9997]

Re: index replication in custom index

adonio — Tue, 09 May 2017 12:14:53 GMT

what are your replication factor and search factor?
do you see the index configuration in all indexers? IDX1, IDX2 etc?
how can you tell your data is served only from IDX2?
try and search: index = dap splunk_server = IDX4 any results?

Re: index replication in custom index

Prakhar_shukla — Tue, 09 May 2017 16:26:44 GMT

Hello Adonio,

Rep factor is 2. Search factor is 2. When I am trying to do search index=dap. Results are only coming from the IDX2 host where I forwarded it from HF. Index configuration is same in all indexer as pushed the config file from cluster master.

Re: index replication in custom index

adonio — Tue, 29 Sep 2020 13:59:55 GMT

did you try the search?
index = dap splunk_server = IDX4
can you check for errors in internal index?
index = _internal log_level - ERROR OR log_level = WARN*

Re: index replication in custom index

shaskell_splunk — Tue, 09 May 2017 17:34:55 GMT

Have a look here:

http://docs.splunk.com/Documentation/Splunk/6.6.0/Indexer/Configurethepeerindexes#The_indexes.conf_repFactor_attribute

You need to specify the following setting for your index in order to have it replicate to other peers in the cluster.

repFactor = auto

Re: index replication in custom index

adonio — Tue, 09 May 2017 17:53:38 GMT

I have to remind myself to never assume.
better make this setting global to avoid future problem as yo encountered now