Deployment Architecture

Can't add a search peer to a cluster search head

responsys_cm
Builder

We're building out a new cluster and want to keep our old data accessible. Since there is no way to forward all of our old data to the new cluster, the recommendation of support was to add the old server as a search peer on the cluster search head.

When I go in to the GUI on the search head and go to the distributed search section, the "Add" button isn't there. I can't add the old server to the cluster because the disk and directory structure is too different.

Do I have to remove the search head from the cluster and then just configure distributed search to search the three cluster indexes and the old server?

Thanks.

Craig

0 Karma

grijhwani
Motivator

Not necessarily. You could, in fact, install a second instance of Splunk onto one of the new machines, with storage for the old logs and address it that way. (You would have to set Splunk up with non-standard ports - e.g. 9089 for management.) However, that still does not address your problem directly, because you would have to add the second instance of Splunk as a separate searchable indexer to the search head. But it us something to consider if you are concerned about the longevity of the legacy hardware.

0 Karma

responsys_cm
Builder

I want to be able to search the data on my old Splunk server and the data on the new cluster. A month from now if I want to run an annual report, I need to be able to search both my old Splunk server's data and the new cluster's data. And I don't want to have to copy my old indexes to one of the cluster indexers. I'd have to rename all of those indexes and then tweak all of the searches that have a "index=xyz" in it...

0 Karma

bmacias84
Champion

What do you mean? Do you still want your old indexes search able?

0 Karma

responsys_cm
Builder

Is it possible to make my old Splunk server a search head for the cluster? Will it then be able to search its own local indexes and the remote ones?

0 Karma

bmacias84
Champion

Currently you are not able to add a none clustered indexer to an to a distributed clustered environment. I have talked to others and the suggested work around is to turn your nonclustered indexers into a cluster of 1. At this point you should be able to added a stand alone indexer as a search peer. I have never done this.

0 Karma

grijhwani
Motivator

Are you sure your Enterprise licence is properly configured and available to the search head?

What is in your $SPLUNK_HOME/etc/system/local/distsearch.conf (or indead default/distsearch.conf)?

What do Splunk's internal logs have to say on the matter? ($SPLUNK_HOME/var/log/splunk/...)

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...