Has anyone written a bash script to install splunkforwarder on a linux server?

[sphiwee]

has anyone written a bash script to install splunkforwarder on a linux server? or is it impossible due to having to enter admin and password and also having to use different users while installing ?

[pc1]

This is an old version of a script I use to install splunk forwarder on Linux (ubuntu) servers and connect up to a splunk enterprise instance and deployment server. In case this may be useful to anyone who comes across this thread in the future 

#!/bin/bash
set -o errexit
# This will delete the installation of splunk forwarder if there already is one installed. allows you to run script to reinstall
sudo rm -f /opt/splunkforwarder

# Edit this section to get the most recent up to date wget command for downloading splunk forwarder
sudo wget -O splunkforwarder-8.2.0-e053ef3c985f-Linux-x86_64.tgz
'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=8.2.0&product=universalforwarder&filename=spl$
sudo tar xvzf splunkforwarder-8.2.0-e053ef3c985f-Linux-x86_64.tgz -C /opt
cd /opt/splunkforwarder/bin

# default username is admin
# generates random password for admin account
sudo ./splunk start --accept-license --no-prompt --gen-and-print-passwd
#enables start on boot
sudo ./splunk enable boot-start

#Adds the instance of where you are going to be forwarding your logs
cd /opt/splunkforwarder/etc/system/local
sudo touch outputs.conf
echo "" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "[tcpout]" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "defaultGroup = default-autolb-group" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "[tcpout:default-autolb-group" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "disabled = false" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "server = NAME_OF_YOUR_SPLUNK_SERVER:9997" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf
echo "[tcpout-server://NAME_OF_YOUR_SPLUNK_SERVER:9997]" | sudo tee -a /opt/splunkforwarder/etc/system/local/outputs.conf

#adds the monitoring of var/log/syslog (relatively important for monitoring linux (ubuntu) servers. optional however and can be configured
from a deployment server)
sudo touch inputs.conf
echo "[monitor://var/log/syslog]" | sudo tee -a /opt/splunkforwarder/etc/system/local/inputs.conf
echo "disabled = 0" | sudo tee -a /opt/splunkforwarder/etc/system/local/inputs.conf

#adds the deployment server for managing the newly created instance (optional but defnitely should use a deployment server)
sudo touch deploymentclient.conf
echo "[deployment-client]" | sudo tee -a /opt/splunkforwarder/etc/system/local/deploymentclient.conf
echo "[target-broker:deploymentServer]" | sudo tee -a /opt/splunkforwarder/etc/system/local/deploymentclient.conf
echo "targetUri = NAME_OF_DEPLOYMENT_SERVER:8089" | sudo tee -a /opt/splunkforwarder/etc/system/local/deploymentclient.conf

#reboots splunk to save changes
cd /opt/splunkforwarder/bin
sudo ./splunk restart

[PickleRick]

Let me point out some flaws with your script.

1) Unconditional removal of /opt/splunkforwarder can be a bit hasty. Especially if you're using TLS and have crypto material stores somewhere in $SPLUNK_HOME

2) You're doing way too many things with sudo. Sudo should be used only if it's absolutely necessary. wget with elevated privileges? What for?

3) You're wgetting the installation archive into current directory. It'd be more elegant to create a temporary directory.

4) Instead of multiple echos you can simply do one cat with here-document.

5)  You're leaving the installation archive behind.

6) The script could be parametrised.

7) You're happily ignoring randomly generated admin credentials. You might need them later.

😎 You can use splunk commands to modify configuration instead of creating the files by hand.

[pc1]

I said it was old. 😆If it works it works 

1) Idk what you mean by crypto materials. bitcoin?
It should probably have a sudo ./splunk stop command before deleting the directory but idk what else you would want

2) wget doesn't need sudo? idk 

3) so essentially cd /opt && wget install splunk && then delete the install file ?

4) yeah ur right

5) i guess it would be better practice to do that. i was just copy and pasting from my notes to try and help

6) it could but i just wrote the variables in caps to be changed since the point of my script when i wrote it is to have Zero user interaction

7) again I want zero user interaction so randomly generating them into the void works for me since you can just manually edit and file in the future. imo they aren't needed or at least I haven't seen a case where i need admin credentials on a forwarder 

[PickleRick]

First things first - it's not to say that the script does 't work or anything like that. It's just that I'm already old and grumpy 😉 and like well-written stuff so I point out what can be done better.

1) No. I mean private keys/certs for TLS. If it was my script and it was meant to be universal I'd do a conditional check for existence of old forwarder and a command-line switch to force removal in case of pre-existing dir.

2) Sure it doesn't it simply connects to a server and pulls a file from there. Where are elevated privileges needed here? Nowhere. You just need to be able to write a file in the destination  directory.

3) The "pretty" solution would be to use mktemp to create a temporary directory (probably somewhere in /tmp) and delete it at the end of the script. Bonus points for capturing error states and removing the package on abnormal exit (but to be quite honest I'm usually too lazy to implement it myself).

7) btool might need them (especially if you want to run it without direct sudo/su to the user running the forwarder. Also listing inputs and monitor states. Sometimes these credentials are nice to have.

[pc1]

@PickleRick  So I did some tweaking with your comments and landed on this. However... I am running into an issue with enabling boot-start or doing any of the ./splunk restart/stop/start commands. Before running it outputs: "Warning: Attempting to revert the SPLUNK_HOME ownership" and "Warning: Executing "chown -R root /opt/splunkforwarder" before the ./splunk restart command - and then the restart command hangs and never finishes. Can't really seem to figure out why this is happening or if its something i did that messed up the filesystem of the desktop I am working on. Let me know what you think 🙂

#!/bin/bash

set -o errexit 



# User input. Comment out this section and replace with $serv and $depserv with hardcoded variables below if you choose to omit

# You also may need to change the port #'s if you chose to not use the splunk default ports in your environment



read -p "Please enter the name of Splunk server you will be forwarding your logs to: " serv

if ping -c 1 "$serv" &> /dev/null

then

    echo "Server Succesfully Reached"

else

    echo "Server cannot be reached"

    exit

fi



while true; do

    read -p "Do you have a Splunk Deployment Server? (y/n) " dep

    if [[ $dep = "y" ]]; then

        read -p "Enter deployment server name: " depserv

        if ping -c 1 "$depserv" &> /dev/null

        then

            echo "Server Succesfully Reached"

            break

        else

            echo "Server cannot be reached"

            exit

        fi



   elif [[ $dep = "n" ]]; then

        echo "User does not have a deployment server"

        break

   else

        echo "Please enter 'y' for yes or 'n' for no: "

   fi

done



# This will check for/delete the installation of splunk forwarder if there already is one installed. allows you to run script again to reinstall

FILE=/opt/splunkforwarder

if [ -d "$FILE" ]; then

    while true; do

    read -p "A version of Splunk is already installed. Do you wish the overwrite? (y/n) " over

        if [[ $over = "y" ]]; then

            sudo /opt/splunkforwarder/bin/splunk stop

            sudo rm -rf /opt/splunkforwarder

            break

        elif [[ $over = "n" ]]; then

            echo -e "\nUser chose to not overwrite existing version of Splunk\n"

            exit

        else

            echo "Please enter 'y' for yes or 'n' for no: "

        fi

    done

fi



# Edit this section to get the most recent up to date wget command for downloading splunk forwarder 
# Splunk version 9.0

temp_dir=$(mktemp -d)

cd $temp_dir

wget -O splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz "https://download.splunk.com/products/universalforwarder/releases/9.0.0/linux/splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz" &> /dev/null

if [[ "$?" != 0 ]]; then

        echo -e "Failed to download file. Exiting script.\n"

        rm -r $temp_dir

        exit

else

        echo -e "\nSplunk files successfuly downloaded\n"

fi

tar xvzf splunkforwarder-9.0.0-6818ac46f2ec-Linux-x86_64.tgz -C /opt

rm -r $temp_dir



# Default username is admin

# Generates random password for admin account. Random password can be found in script output if you wish to take note of it

cd /opt/splunkforwarder/bin

sudo ./splunk start --accept-license --no-prompt --gen-and-print-passwd

#sudo ./splunk enable boot-start seems to not be working here. splunk must be stopped for it to be enabled



#Adds the instance of where you are going to be forwarding your logs

cd /opt/splunkforwarder/etc/system/local

sudo touch outputs.conf

echo -e "Outputs.conf \n"

sudo tee -a outputs.conf << END 

[tcpout]

defaultGroup = default-autolb-group



[tcpout:default-autolb-group]

disabled = false

server = $serv:9997



[tcpout-server://$serv:9997]



END



# Adds the monitoring of var/log/syslog (relatively important for monitoring linux (ubuntu) servers)

# This is optional. The inputs you choose to monitor should be configured from a deployment server 

# Comment out if you choose it is not needed

sudo touch inputs.conf

echo -e "Inputs.conf \n"

echo "[monitor://var/log/syslog]" | sudo tee -a /opt/splunkforwarder/etc/system/local/inputs.conf

echo "disabled = 0" | sudo tee -a /opt/splunkforwarder/etc/system/local/inputs.conf



# Adds the deployment server for managing the newly created instance (optional but defnitely should use a deployment server)
if [[ $dep = "y" ]]; then

    sudo touch deploymentclient.conf

    echo -e "\nDeploymentclient.conf \n"

    sudo tee -a deploymentclient.conf << END

[deployment-client]

[target-broker:deploymentServer]

targetUri = $depserv:8089

END

fi



echo -e "\nEnd of file changes \n"



#reboots splunk to save changes

#sudo ./opt/splunkforwarder/bin/splunk stop

#sudo /opt/splunkforwarder/bin/splunk enable boot-start

#sudo ./opt/splunkforwarder/bin/splunk start

cd /opt/splunkforwarder/bin

sudo ./splunk restart

[PickleRick]

Well, you did the script completely differently from what I'd do 🙂

I prefer non-interactive scripts for such uses - possibly with lotsa commandline switches if I want to run them in batch across many systems so I don't have to react to everything every time. But that's up to personal taste.

About the overwriting - I don't think I ever deployed forwarder from the tgz file (I usually use the rpm packages) but I suppose the files within the archive might indeed have a "wrong" ownership and you would have to do chown /opt/splunk before running the forwarder so you don't get in trouble later on.

[pc1]

This is great constructive feedback of several things I didn't know about. Thank you 🙌

[nikhilmfwd]

hi team,

Can you please provide shell script  to install forwarder into server. Basically i have a task to automate same.

[richgalloway]

Since this thread is 2 years old with an accepted solution, you should post a new question.

That said, have you tried this one? https://community.splunk.com/t5/Getting-Data-In/Simple-installation-script-for-Universal-Forwarder/m...

[thambisetty]

It’s not impossible as you don’t require to set admin password while installing. You can automate this one completely.

I will share script soon.

[sphiwee]

Thank you, cant wait to see it

[thambisetty · Accepted solution]

This is applicable for version 7.2.2 later.

you should run below commands with sudo user

useradd splunk
tar splunkbinary.gz -C /opt
chown -R splunk:splunk /opt/splunkforwarder
/opt/splunkforwarderk/bin/splunk enable boot-start -systemd-managed 0 -user splunk --no-prompt --accept-license
sudo -u splunk /opt/splunkforwarderk/bin/splunk start

[sphiwee]

and when do i add admin and password?

[thambisetty]

Hope you will manage deployment clients(where uf is installed) with deployment server. If yes, you don’t require to set the password at all.

[sphiwee]

okay, so when I add a log to monitor it wont require a password

[thambisetty]

that's why I have asked you, I hope you manage this client from Deployment server.

It will prompt for password if you add from CLI using splunk command.

you can update inputs.conf  to avoid prompting for password.

[sphiwee]

Yes I'm using CLI

[isoutamo]

With sensible you could also set admin passwords when you installing it.

Here is Splunk's own ansible which they are actively updated.

• https://github.com/splunk/splunk-ansible/tree/master

And some other playbooks:

• https://github.com/AustinCloudGuru/ansible-role-splunk-forwarder
• https://www.linuxsysadmins.com/splunk-forwarder-installation-using-ansible/

r. Ismo

[masonmorales]

See also: Splunk's official Ansible role for VMs and bare metal

[richgalloway]

See https://docs.splunk.com/Documentation/Forwarder/8.0.5/Forwarder/Installanixuniversalforwarderremotel...