- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

marycordova

SplunkTrust
05-23-2023
08:00 PM
Is there a basic cheatsheet for setting up a new small scale distributed deployment?
@marycordova
1 Solution
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

marycordova

SplunkTrust
05-23-2023
08:37 PM
- Here are the basic steps for setting up a new deployment.
- Build your Deployment Server first through step 8. You can use the WebUI on the Deployment server to upload your license file.
- Build your indexers, heavy forwarders, & search head with the same steps adding items 9-13.
- Be sure you also configure your Deployment server with your indexes, SSL, and forward the internal logs to your indexers.
- These steps were done on Ubuntu so commands may vary slightly on other *nix flavors. You may have other ways (systemd) to do some of the *nix admin such as the limits and THP.
- login via cli & elevate to root
- increase system limits
vi /etc/security/limits.conf
* hard nofile 64000
* hard nproc 16000
* hard fsize -1
- disable THP
- https://www.mongodb.com/docs/manual/tutorial/transparent-huge-pages/ System V Init (service)
vi /etc/init.d/disable-transparent-hugepages
#!/bin/bash
### BEGIN INIT INFO
# Provides: disable-transparent-hugepages
# Required-Start: $local_fs
# Required-Stop:
# X-Start-Before: splunk
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Disable Linux transparent huge pages
### END INIT INFO
echo 'never' | tee /sys/kernel/mm/transparent_hugepage/enabled > /dev/null
echo 'never' | tee /sys/kernel/mm/transparent_hugepage/defrag > /dev/null
chmod 755 /etc/init.d/disable-transparent-hugepages
/etc/init.d/disable-transparent-hugepages start
update-rc.d disable-transparent-hugepages defaults
- reboot, login, & elevate to root
- create the splunk user
useradd -m splunk
- install splunk & configure to run as non-root "splunk" user at system boot
cd /opt
wget -O splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz "https://download.splunk.com/products/splunk/releases/9.0.4.1/linux/splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz"
tar zxfv splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz
chown -R splunk:splunk /opt/splunk
/opt/splunk/bin/splunk enable boot-start -user splunk
- reboot, login, & elevate t root
- verify configs
ulimit -a
cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag
ps -ef | grep splunk
- switch to splunk user
su - splunk
- add deploymentclient.conf
vi /opt/splunk/etc/system/local/deploymentclient.conf
[deployment-client]
[target-broker:deploymentServer]
targetUri = https://deploymentserver.yourdomain.com:8089
- add the Splunk license, restart Splunk, & check licensing
/opt/splunk/bin/splunk edit licenser-localpeer -manager_uri 'https://deploymentserver.yourdomain.com:8089'
/opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk list licenser-localpeer
- add to appropriate Deployment Server Class(es) & deploy appropriate apps
- enable (SH, HF, DS) or disable (indexers) the WebUI
- enable cooked Splunk port 9997 inputs on the indexers
- forward _* internal logs to the indexers
- define indexes
- enable SSL
- WebUI, 8089 mgmt, 9997 cooked input
- enable email (SMTP) for SearchHead only via the SH WebUI
@marycordova
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

marycordova

SplunkTrust
05-23-2023
08:37 PM
- Here are the basic steps for setting up a new deployment.
- Build your Deployment Server first through step 8. You can use the WebUI on the Deployment server to upload your license file.
- Build your indexers, heavy forwarders, & search head with the same steps adding items 9-13.
- Be sure you also configure your Deployment server with your indexes, SSL, and forward the internal logs to your indexers.
- These steps were done on Ubuntu so commands may vary slightly on other *nix flavors. You may have other ways (systemd) to do some of the *nix admin such as the limits and THP.
- login via cli & elevate to root
- increase system limits
vi /etc/security/limits.conf
* hard nofile 64000
* hard nproc 16000
* hard fsize -1
- disable THP
- https://www.mongodb.com/docs/manual/tutorial/transparent-huge-pages/ System V Init (service)
vi /etc/init.d/disable-transparent-hugepages
#!/bin/bash
### BEGIN INIT INFO
# Provides: disable-transparent-hugepages
# Required-Start: $local_fs
# Required-Stop:
# X-Start-Before: splunk
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Disable Linux transparent huge pages
### END INIT INFO
echo 'never' | tee /sys/kernel/mm/transparent_hugepage/enabled > /dev/null
echo 'never' | tee /sys/kernel/mm/transparent_hugepage/defrag > /dev/null
chmod 755 /etc/init.d/disable-transparent-hugepages
/etc/init.d/disable-transparent-hugepages start
update-rc.d disable-transparent-hugepages defaults
- reboot, login, & elevate to root
- create the splunk user
useradd -m splunk
- install splunk & configure to run as non-root "splunk" user at system boot
cd /opt
wget -O splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz "https://download.splunk.com/products/splunk/releases/9.0.4.1/linux/splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz"
tar zxfv splunk-9.0.4.1-419ad9369127-Linux-x86_64.tgz
chown -R splunk:splunk /opt/splunk
/opt/splunk/bin/splunk enable boot-start -user splunk
- reboot, login, & elevate t root
- verify configs
ulimit -a
cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag
ps -ef | grep splunk
- switch to splunk user
su - splunk
- add deploymentclient.conf
vi /opt/splunk/etc/system/local/deploymentclient.conf
[deployment-client]
[target-broker:deploymentServer]
targetUri = https://deploymentserver.yourdomain.com:8089
- add the Splunk license, restart Splunk, & check licensing
/opt/splunk/bin/splunk edit licenser-localpeer -manager_uri 'https://deploymentserver.yourdomain.com:8089'
/opt/splunk/bin/splunk restart
/opt/splunk/bin/splunk list licenser-localpeer
- add to appropriate Deployment Server Class(es) & deploy appropriate apps
- enable (SH, HF, DS) or disable (indexers) the WebUI
- enable cooked Splunk port 9997 inputs on the indexers
- forward _* internal logs to the indexers
- define indexes
- enable SSL
- WebUI, 8089 mgmt, 9997 cooked input
- enable email (SMTP) for SearchHead only via the SH WebUI
@marycordova
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

marycordova

SplunkTrust
05-23-2023
09:02 PM
I'll try to add a little more detail here when I get a chance:
- add to appropriate Deployment Server Class(es) & deploy appropriate apps
- enable (SH, HF, DS) or disable (indexers) the WebUI
- enable cooked Splunk port 9997 inputs on the indexers
- forward _* internal logs to the indexers
- define indexes
- enable SSL
- WebUI, 8089 mgmt, 9997 cooked input
@marycordova
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
inventsekar

SplunkTrust
05-23-2023
08:36 PM
Hi @marycordova ...
As per my knowledge there are no cheetsheet kind of documents..
but the Splunk documentation is very detailed and it will give us all the info required.
Please install SH, indexers, as per this document:
https://docs.splunk.com/Documentation/Splunk/9.0.4/Installation/InstallonLinux
then you can configure the indexer cluster and search head cluster as per this document:
https://docs.splunk.com/Documentation/Splunk/9.0.4/Deploy/SHCwithindexers
hope this helps.. thanks.
thanks and best regards,
Sekar
PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
Sekar
PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !
