topic Re: Can an alert be run from a specific Search Head in a clustered environment? in Splunk Search

Can an alert be run from a specific Search Head in a clustered environment?

nicofantinato — Mon, 02 Nov 2020 17:03:29 GMT

Hi all,

we have a Splunk Enterprise clustered environment, with a cluster of 3 search heads.

For many reasons, a lookup file is updated once a day in only one of these search heads (the first one).

To update this lookup file also in the other two search heads, we set up a scheduled search with the following string:

| inputlookup my_lookup_table.csv | outputlookup my_lookup_table.csv

Since if this search is run from a different search head than the number one the lookup is not updated, is it possible to run it always from the same search head? I know we could send the lookup via SFTP to the other search heads servers, but if possible we'd like to avoid it.

Thanks in advance.

Re: Can an alert be run from a specific Search Head in a clustered environment?

richgalloway — Mon, 02 Nov 2020 17:39:12 GMT

The SHC captain decides which member will run each scheduled search. There is no provision for overriding that decision.

How is the lookup file updated in the first place? Could that utility also update the other SHC members?

Re: Can an alert be run from a specific Search Head in a clustered environment?

Azeemering — Mon, 02 Nov 2020 17:56:03 GMT

What am I missing here? If you have clustered search heads you also should have configured cluster replication. For a search head cluster to function properly, its members must all use the same set of search-related configurations.

https://docs.splunk.com/Documentation/Splunk/8.1.0/DistSearch/HowconfrepoworksinSHC

But if you want to run a search from a specific search head you could theoretically configure all the other search heads to only run ad hoc searches. ‌‌In server.conf add the following 😂

[shclustering]
adhoc_searchhead = true

Re: Can an alert be run from a specific Search Head in a clustered environment?

nicofantinato — Tue, 03 Nov 2020 08:29:43 GMT

Hi Azeemering. Yep, cluster replication is configured, but if you copy a lookup file under $SPLUNK_HOME/etc/apps/app_name/lookups it is updated only on that specific search head, replication is done only if click Save button from web console... or at least this is the behaviour we observed in our environment.

Re: Can an alert be run from a specific Search Head in a clustered environment?

nicofantinato — Tue, 03 Nov 2020 08:33:42 GMT

Hi richgalloway.

"The SHC captain decides which member will run each scheduled search. There is no provision for overriding that decision." that's what we were afraid of.

The lookup comes form a curl command, a script launches the command once a day in only one of the search heads. Security guys want us to do this way.