Thanks, I think https://docs.splunk.com/Documentation/Splunk/9.2.1/InheritedDeployment/Ports is the one included? ... View more

Updated link: https://docs.splunk.com/Documentation/Splunk/latest/Admin/Configurationparametersandthedatapipeline ... View more

You may take a look at below to see the detailed info on changing Splunk DB location and copying the data from existing directory to new. https://pandeysumitsolutions.blogspot.com/2023/05/changing-splunk-db-location-if-our.html Blog link - http://sumitpandey.co.in  ... View more

Updated link: https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Whathappenswhenamanagernodegoesdown ... View more

If you are removing index definition from CM’s indexes.conf it just removes that definition from peers’ configuration, BUT it don’t remove actual index directories/events from disk. Those you must remove manually from every nodes.  ... View more

What are cyber threats to Splunk network ports. Can someone tell ... View more

Hi, Did you find a fix for this issue? ... View more

Are you saving the frozen buckets to a local directory first with coldtofrozendir? I pushed out an app with just this script to my indexers, and im not seeing anything in my s3 bucket.  When i run the script manually, I get  bin>./smartstore2frozen.sh ./smartstore2frozen.sh: line 5: $1: unbound variable   When I comment out 'set -u'  I get bin>./smartstore2frozen.sh date: invalid date ‘@’   Heres the script. The only thing i modified is that I removed the proxy portion and changed the bucket name #!/bin/bash set -e set -u bucket=$1 instance=$(hostname -s) region=us-east-1 s3bucket=<my s3 bucket> NOW=$(date +"%Y-%m-%d") LOG=/opt/splunk/var/log/splunk/smartstore2frozen-${NOW}.log #Gets index name and warm bucket name from path passed by splunk index=$(echo $1 | cut -f7 -d"/") warm=$(echo $1 | cut -f9 -d"/") #Converts the epoch time from the warm bucket name to EST startEpoch=$(echo $warm | cut -f3 -d"_") endEpoch=$(echo $warm | cut -f2 -d"_") startDate=$(date -d @$startEpoch '+%m_%d_%Y') endDate=$(date -d @$endEpoch '+%m_%d_%Y') #Sets AWS Sginature Version - Needed for S3 KMS-SSE aws configure set s3.signature_version s3v4 #Creates log file touch ${LOG} echo "bucket to move: " $bucket >> $LOG #Copies bucket to S3 and logs the output along with timestamps /usr/bin/aws s3 cp ${bucket} s3://${s3bucket}/frozen/${index}/${startDate}_to_${endDate}/${warm} --recursive --region ${region} 2>&1 | tr "\r" "\n" > >(awk '{print strftime("%Y-%m-%d:%H:%M:%S ") $0}' >> $LOG) ... View more

Yes, and the same partition on the storage. ... View more

So what about Cluster Master's disk requirements? Let say there are two indexers need to be managed by cluster masters? ... View more

What you have mentioned does match the documentation/training... ... View more

Thank you for your answer. In conjunction with the other answer, it helped me gain useful clarity regarding some ambiguous material in the current docs. I have begun improving that material based on the answers, and hope to make improvements to other licensing material in the coming days and weeks. ... View more

When you add each indexer as a search peer in distsearch.conf (or the GUI), vs joining the SH/SHC to a index cluster, you are essentially eliminating the mapreduce functionality of distributed search and the index cluster. This is because the SH/SHC no longer knows its searching a cluster, nor does it know where the searchable buckets are. So its essentially distributed to all indexers, and awaiting response. So extra compute... Rebuild your SHC to connect to the cluster master node and let search run properly. That should alleviate load. After that, you need to check through search log and determine where your searches are slowing down. E.g., is it indexer side disk or memory, or SH. From there, adjust resources as required.. ... View more

Thank you Steve G. for your answer. I found that there is no way to setup a multisite indexer cluster with some indexes replicated on all sites, and some just for a particular indexer cluster described in the documentation. Having an index on a single peer is supported though ( http://docs.splunk.com/Documentation/Splunk/6.6.1/Indexer/Managesinglepeerconfigurations#Add_an_index_to_a_single_peer ) So what I ended up doing to get a one-site-only index is to: Create a new file on the index master: etc/slave-apps/_cluster/local/site1-big-indexes.conf which is part of the configuration bundle. Then, I can use the usual: splunk apply configuration-bundle which ensures that new revision of that file makes it to all indexers (or none). I do get a warning: [Not Critical]No spec file for: /indexmaster/etc/master-apps/_cluster/local/site1-big-indexes.conf I went on every indexer on that particular "big" site and added a symbolic link: cd etc/system/local; ln -sf ../../slave-apps/_cluster/local/site1-big-indexes.conf indexes.conf The only thing which is not automatically taken care of is the rolling restart if I update the site-only site-big-indexes.conf I just manually issue a: splunk rolling-restart cluster-peers for that. This seems to work, I now have indexes replicated on all sites and some that are replicated on one site only. Thanks, ... View more

Thanks for the clarification on this! Yes I was looking at the spec file primarily and that's where I started getting a bit confused by the description of the attribute. ... View more

I downvoted this post because the problem that the user had was not with establishing connectivity between search heads and search peers. rather, the problem was with the query used to test connectivity. the accepted answer provided a procedure tht, if followed by others, would cause them to completely rebuild their search head cluster, among other issues. this answer should be removed, as it could easily mislead anyone who does have search peer connectivity issues. ... View more

-- This was confirmed in a lab by pointing the outputs to lab indexers while using a different set of indexers as search peers. All DM buckets are built fine on search peers and are usable for pivot searches. All DM buckets are segregated by search head group (GUID) so the are isolated from impacting other search clusters you just add up more storage for each search group ( standalone/pool/cluster) and more running searches on your search peers. ... View more

No, only incoming data counts against the license. Replicated data does not. See: http://docs.splunk.com/Documentation/Splunk/6.4.3/Indexer/Systemrequirements#Licensing_information ... View more

It is correct that you do not need to set up the deployer connectivity until the first time that you need to use the deployer. However, the deployer is the only supported method for deploying apps and other configurations to cluster members. So, sooner or later, you will need to set up the deployer. You cannot deploy an app until you do so. For this reason, we recommend that you configure the deployer and connectivity to the deployer at the same time that you initially configure the cluster itself. There is usually no reason to wait until later. ... View more

Thanks, a little annoying that the complete documentation is split across two places... Thanks for the assistance!! ... View more

Great. Thanks! ... View more

For anyone who cares (or next time I run into this issue), ... I found and resolved my issue. I missed setting up a TCP Input on one of the 4 peer nodes. (This is why automation rocks, and doing stuff by hand is evil). I found that if I hit the /services/cluster/config endpoint on all the peer nodes, the one that was causing issues was returning ? for forwarderdata_rcv_port . Apparently the cluster master sends this bogus value to the UF's via the /services/indexer_discovery endpoint. Whoops. Find it in splunk like this: | rest services/cluster/config | search mode=slave forwarderdata_rcv_port="?" ... View more