topic Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue? in Deployment Architecture

Search head cluster dilemma -- Is there a way to reverse this configuration issue?

xsstest — Thu, 28 Sep 2017 02:46:49 GMT

hi everyone:

I seem to have made a mistake on the cluster. I wanted to add a lookup table in the lookups directory of search app ($SPLUNK_HOME/etc/apps/search/lookups on everyone cluster member). In order to make all the search head (4 search head) have the same configuration. I did the following steps：

Step 1: copy the search app of one of the search heads to deployer
Step 2: then I added a lookup table in the $SPLUNK_HOME/etc/shcluster/apps/search/lookups/ directory on deployer.
Step 3: I pushed the configuration changes to the cluster members through the splunk apply shcluster-bundle -target https://xxxx:8089 command

I thought that would allow all members to have the same lookup table, Prior to this, all knowledge objects were created through GUI

But then I found that I could not delete my own fields, alerts and other knowledge objects.

As an administrator, I can't delete my own knowledge objects, but about 1% of the knowledge objects can be deleted

Did i make a mistake on the cluster?So now, how do I rescue my search header cluster and get them back to normal?

may you tell me the steps?

See screenshot 1:

Two new directories( default.old.date-bundle id ) are added to the search head ,( because I pushed twice bundles through the deployer. ).

See screenshot 2:

I am copying the entire search app ($SPLUNK_HOME/etc/apps/search) to the deployer. And then configure the changes. Finally pushed to the cluster member

Why i would use the wrong method? I always thought that only put lookup table in the lookups directory of search app, then can call the lookup table on the Search APP(search & Reporting).If the lookup table put other app directory , then can not call the lookup table on the Search APP (search & Reporting).So my idea is wrong?

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

gjanders — Fri, 29 Sep 2017 14:52:14 GMT

There appears to be some confusion here, if i have interpreted your post correctly you have pushed the lookup file from the deployer to the search heads in a cluster and now you cannot edit the lookup on the search heads?

You may want to consider installing the lookup file editor as this might make it easier for you to add lookups via the GUI.
Or use the built-in Splunk lookup functionality to upload your lookup and change the sharing on it so you can access it in all applications if that is what you are trying to do.

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

xsstest — Wed, 11 Oct 2017 15:40:44 GMT

Thank you, I understand now. I can create an app on the deployer. then put the lookup table in the app directory $SPLUNK_HOME$ /splunk/etc/shcluster/apps/myapp_name/test.csv, then push it to all the search header members. I set the lookup table to global sharing through WEBUI.

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

xsstest — Wed, 11 Oct 2017 15:43:01 GMT

One more question.
If one of the members uploads a lookup table via webui, will the other members copy each other?

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

gjanders — Tue, 29 Sep 2020 16:13:39 GMT

Yes, lookup tables for example as per the How configuration changes propagate across the search head cluster do replicate within the search head cluster.

Also under your apps/myapp_name/ you should have a default or local directory where you put the relevant files (myapp_name/default/test.csv for example)
When pushed to the search head members the files will end up in myapp_name/default/... (this way you can override the file on the search head itself)

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

bandit — Wed, 18 Jul 2018 14:56:21 GMT

Good related post for Enterprise Security running in a search cluster with the same dilema. https://answers.splunk.com/answers/498425/how-do-you-update-lookups-on-a-shc-while-running-s.html

Re: Search head cluster dilemma -- Is there a way to reverse this configuration issue?

woodcock — Wed, 18 Jul 2018 15:09:05 GMT

If you use the GUI or the Lookup File Editor app (https://splunkbase.splunk.com/app/1724/), these changes will be synchronized across the cluster. Do not use the Deployer for a simple Lookup File change. You are risking big trouble if you do.