Hi,
I have 3 different instances that are totally separate.
Which architecture and configuration is the best to collect data from mix of architectures?
Thanks
Using Splunk to search other Splunk instances can be accomplished as noted via distributed search.. There are a few different approaches to this depending on the architecture you are connecting to and want to search..
1) Connecting a SH to an indexer cluster -- You need to connect the standalone SH to the cluster by joining the cluster via adding the Cluster Master (CM) as the search peer, or join the cluster via . This will enable the SH to perform standard search against the cluster. If you add individual indexers as search peers, you can also search this data but this isn't the correct way to search against a cluster though, so be cautious and follow the process listed above and here : https://docs.splunk.com/Documentation/Splunk/8.0.3/Indexer/Configureclusteredandnonclusteredsearch
2) Connecting to a "Standalone" Indexer. This process is different then joining a cluster. You have to add the Indexer as a Distributed Search Peer. For the initial setup, this will require that you have an admin user account on the Indexer to join it as a peer. Once joined successfully you should see successful status.
So be aware there are differences depending on what you are connecting to. E.g., a Cluster or a standalone indexer.
Regarding the 303 error you are getting, this is probably related to adding the clusters indexers as search peers and not joining the cluster properly. Make sure you go through the proper process as documented above.
Using Splunk to search other Splunk instances can be accomplished as noted via distributed search.. There are a few different approaches to this depending on the architecture you are connecting to and want to search..
1) Connecting a SH to an indexer cluster -- You need to connect the standalone SH to the cluster by joining the cluster via adding the Cluster Master (CM) as the search peer, or join the cluster via . This will enable the SH to perform standard search against the cluster. If you add individual indexers as search peers, you can also search this data but this isn't the correct way to search against a cluster though, so be cautious and follow the process listed above and here : https://docs.splunk.com/Documentation/Splunk/8.0.3/Indexer/Configureclusteredandnonclusteredsearch
2) Connecting to a "Standalone" Indexer. This process is different then joining a cluster. You have to add the Indexer as a Distributed Search Peer. For the initial setup, this will require that you have an admin user account on the Indexer to join it as a peer. Once joined successfully you should see successful status.
So be aware there are differences depending on what you are connecting to. E.g., a Cluster or a standalone indexer.
Regarding the 303 error you are getting, this is probably related to adding the clusters indexers as search peers and not joining the cluster properly. Make sure you go through the proper process as documented above.
So, in this architecture, are standalone Splunk server able to connect both indexer cluster SH ( connect to indexer master too) and standonale other Splunk Server as Distributed Search at the same time?
You mentioned OR in your description.
Thanks
Stand alone Splunk Enterprise instances (both acting as a search head, or indexer) can search both other standalone indexers and also indexing clusters. The main point to this is though, that the method for connecting to these are different.
At a high level, to connect to a stand alone (non-clustered) indexer, you simply add that indexer as a search peer. The process for joining a cluster is different however, since we have to connect to the Cluster Master (CM) role in order to search the data in that cluster correctly.
The number of peers and clusters a SH can search against is near unlimited. But there are architectural decisions to make when you have multiple deployments. This is mostly around bandwidth and network latency between the SH and peers you are searching. The Deploying and Architecting Splunk manual covers this more in detail ( https://doc.splunk.com)
Thanks alot for the details. Now I am able to add entire cluster using standalone SH successfully.
2 indexers,1 Master,1 SH(Site 1) ---> Central SH(Site 2) <---Standalone SH (Site 3)
Hi,
Based on my understanding you have 3rd instance as standalone Search Head and want to search data from 1st standalone splunk instance and 2nd Indexer cluster servers. If that is the case then go through this doc https://docs.splunk.com/Documentation/Splunk/8.0.3/Indexer/Configureclusteredandnonclusteredsearch and configure your 3rd instance.
Thanks for the answer. I checked this out and I have many errors to connect SH for indexer cluster from Single Server SH.
Check credentials/firewalls. Any experience on this?
ERROR DistributedBundleReplicationManager - Unexpected problem while uploading bundle: Unknown write error
ERROR DistributedBundleReplicationManager - Bundle Replication: Problem replicating config (bundle) to search peer ' xxxxxx:8089 ', HTTP response code 303 (HTTP/1.1 303 See Other)
ERROR DistributedBundleReplicationManager - Unable to upload bundle to peer named xxxx with uri=https://xxxxxx:8089.
WARN DistributedPeerManager - Cannot determine a latest common bundle, search may be blocked
Which steps have you performed ?
Connect to Indexer cluster SH via using standalone server. What I realize from @esix_splunk comment below is to do same thing for indexer master too. This could be the issue(?)
As @esix_splunk mentioned and if you follow document properly, you can see that to search data from Indexer Cluster you need to point your Search Head to Cluster Master (No need to add clustered indexer in distsearch.conf on SH). To search data from standalone instance you need to add that instance as search peer on SH(3rd Instance).