We are planning to expand existing Splunk setup.
Present : We have one Splunk indexer (172.16.XX.XX) , we are forwarding data to that indexer and accessing Splunk UI for that server.
Planning : one new indexer in new server (172.16.XX.XX) and one search head in new server (172.16.XX.XX).
Final setup : Two indexers and one search head.
Below are some queries,
• Here, What do we need to install one new indexer and search head ?
• How we can forward data to the new indexer and to the old indexer from all the forwarders ? How we can access Splunk UI for new search head. Any ports or any routes we need to open on new servers ?
• Here, we will forward data to the both new and old indexers , we can access splunk UI for new search head. How we can see the indexing data in Splunk UI ?
Hi @RAYUDU_NARA,
Please find below answers.
autoLB
, data will be distributed among both the indexers and in future if any one indexer will go down then you will lose half of the data so you need to consider this as risk. To avoid this situaltion it will be good to setup Indexer Cluster.http
so you can access splunk search head UI with URL http://<search head FQDN/IP>:8000
To conclude above steps in splunk way, you can read this documentation
I hope this helps.
Thanks,
Harshil
Hi @RAYUDU_NARA,
I don't know if i understood your question correctly.
If you want to change your standalone server role from search head, you can do it via monitoring console of your standalone instance.
1. In Splunk Web, navigate to Monitoring Console > Settings > General Setup.
2. Click Edit on instance that you want to disable as a search head.
3. Click Apply Changes to complete setup.
Kindly go through the below splunk docs it might help you.
https://docs.splunk.com/Documentation/Splunk/7.0.1/DMC/Configureinstandalonemode
Hi @nikita_p
Yes, this is really helpful. Thank you.
I have one more doubt, We want to forward data to the two indexers from all the forwarders. Can you please suggest one good way to configure.
We have license for existing setup. How we can share with new indexer ?
Hey, @RAYUDU_NARA, if @harsmarvania57 answered your question, please remember to "√Accept" the answer to award karma points and to let other Splunkers know it’s a golden answer. We’re hosting a karma point contest, so it’s particularly awesome to up vote on Answers these days. 😄
Hi @RAYUDU_NARA,
Please find below answers.
autoLB
, data will be distributed among both the indexers and in future if any one indexer will go down then you will lose half of the data so you need to consider this as risk. To avoid this situaltion it will be good to setup Indexer Cluster.http
so you can access splunk search head UI with URL http://<search head FQDN/IP>:8000
To conclude above steps in splunk way, you can read this documentation
I hope this helps.
Thanks,
Harshil
Hi Harshil,
It is very clear. Thanks for your help.
Regards,
Rayudu
Hi Harshil,
Now we have Indexer and search head on same server. We need to disable search head on old server and make it new server as a search head for both the indexer. How we can do this ?
Hi Rayudu,
If you want to use new search head with fresh installation and do not want to migrate any reports/alerts , dashboards, field extractions etc. from old search head then you just need to follow last points which I have given in my answer To search data from both indexers on search head , you need to configure search peers (Indexers) in search head. Very good documentation here
If you want to migrate knowledge objects (reports/alerts , dashboards, field extractions etc.) from old search head to new search head then follow last points in my answers and additionally you need to copy
1.) $SPLUNK_HOME/etc/apps/<App name>/local/
and $SPLUNK_HOME/etc/users/<Username>/<App name>/local/
directory from old search head to new search head
2.) $SPLUNK_HOME/etc/apps/<App name>/metadata/local.meta
and $SPLUNK_HOME/etc/users/<Username>/<App name>/metadata/local.meta
directory from old search head to new search head.
At last restart splunk on new search head.
I hope this helps.
Thanks,
Harshil
Hi @harsmarvania57 - Thank you for you help. few more questions again ,
1) For ITSI app, do we have any seperate config to move to new search head.
2) What happens to scheduled searches – do they move to the new search head?
3) What happens to our non-forwarded inputs – eg, REST (lots of these now). Do they stay on their existing node?
Please suggest.
Hi Harshil,
I think this is for to make a new server as a search head. How we can disable search head from the existing setup (stand-alone)
There are no such setting to disable search head on Indexers, however you can remove customised role mapping from authentications.conf and custom roles from authorize.conf.
Also you need to remove knowledge objects (reports/alerts , dashboards, field extractions etc.) which were created by users from that indexer so that it will not run those knowledge objects.
Hi @harsmarvania57 - i accepted your answer that is helpful to me. Thank you.
For more clarity few more doubts, Please suggest.
We want to forward data to the two indexers from all the forwarders. Can you please suggest one good way to configure ? (autoLB or any node configuration)
We have license for existing setup. How we can share with new indexer ?
autoLB is good method to distribute data among Indexers but only drawback in your environment is when one indexer will go down you will lose half of the data.
Another method is to clone data to both the indexers but it will use double license & storage so I'll not prefer that method and stick with autoLB and consider risk as you will lose data when any indexer will be down.
Regarding license to share with another indexer, can you please let us know on which server license is currently installed ?
If we configure auLB frequency method. Data will forward like 40sec to one indexer after 40sec it will forward data to the another indexer. If any one indexer is down, it will forward data to the active indexer. is it correct ?
License:
Now we have license in existing server (indexer and search head is there in that server). after upgrade we will use this server as indexer-1 and will move search head to new server, and indexer-2 also in another new server.
When you configure autoLB at that time if you do not want to lost any data from forwarder to indexer you need to use useACK=true
on forwarder side (Ref doc here), when you use useACK=true
forwarder will wait until Indexer will give acknowledgement back to forwarder that Indexer got data if forwarder will not receive acknowledgement from Indexer in that case forwarder will send same data to another indexer so you will not lose any data during transmission.
Re license: I'll suggest you to move license on Search Head and make search head as license master and both Indexer as license slave. Ref. doc https://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Swapthelicensemaster and http://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Configurealicenseslave
Thank you @harsmarvania75 , will test it and get back to you.
Hi Rayudu,
It will be good to accept my answer so that this question will be closed and other people will able to refer this answer in future.
Thanks,
Harshil