Knowledge Management

Distsearch.conf and other config files overridden after rolling retsart

neeravmathur
Explorer

Hi, 

We have 3 search head in a cluster and 3 indexers in non clustered environment. Whenever we do a rolling restart of the SH, the distsearch.conf in etc/system/local and some lookup csv in some of apps change. It does not happen always but very often. Can anyone help in figuring why this happens and what needs to be corrected. There is no other distsearch.conf anywhere on the SH.

 

Thanks for your help....

Labels (1)
0 Karma

scelikok
Champion

Hi @neeravmathur,

So, I think one of your search heads in the cluster cannot sync with the captain. You can see below document for sync problems;

https://docs.splunk.com/Documentation/Splunk/8.1.2/DistSearch/HowconfrepoworksinSHC#Replication_sync... 

Distributing app via deployer is best practice but I don't think it will help since it will update the files only after apply shcluster-bundle command. The reason I asked this is, a member would have been rejoins to cluster and sync. 

If this reply helps you an upvote is appreciated.
0 Karma

scelikok
Champion

Hi @neeravmathur,

You can check the distsearch.conf file and lookups in your deployer. 

If this reply helps you an upvote is appreciated.
0 Karma

neeravmathur
Explorer

Hi @scelikok

Thanks for your reply. The deployer has no such lookup/distearch file copy. So not really sure from where the SH are picking up the file after rolling restart.

Let me ask a different question-If I use the deployer to deploy an app (with distsearch.conf in it) so even if the file in etc/local gets corrupted, the precedence will always be given to the distsearch.conf in etc/apps/...

Will this work??

Thanks,

Neerav

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.