Simple installation script for Universal Forwarder

[lguinn2]

When you have more than a few forwarders to maintain, it becomes tedious (and error-prone) to install them one-by-one. Using the Deployment Server is great for keeping the configurations up-to-date and consistent.

But I want to script the installation. This isn't really a Splunk problem, it's more of a scripting problem.

Any great scripts out there? All answers welcome, especially Linux and Windows.

[Pranav_Support]

Hello Splunkers! 🎉

You can now easily install Splunk Enterprise and the Universal Forwarder using this handy script. It supports all available versions and can be installed on any Linux distribution. For detailed installation steps, please visit :

https://github.com/PraxisForge/Install_Splunk

Upgrade universal forwarder version (nix) #Splunk Enterprise(nix) #Universal Forwarder(nix) #Upgrade Splunk Enterprise version(nix)

[serjandrosov]

Hello guys,
I made some tweaks for this script.
What's updated:
1. Setting initial user via user-seed.conf to respect splunk v. 7.1+
2. Setting deployment server via the seed app. Using CLI command is not recommended by Splunk PS cause these settings can not be changed from deployment server in the future.
3. Added splunk "runas" user which will be added on a remote host.
4. Added current directory tweak ($DIR) so script can be run by any filesystem path.
Checked in my lab environment with fedora linux server.

#!/bin/sh  
# This EXAMPLE script shows how to deploy the Splunk universal forwarder
# to many remote hosts via ssh and common Unix commands.
# For "real" use, this script needs ERROR DETECTION AND LOGGING!!
# --Variables that you must set -----
# Set username using by splunkd to run.
  SPLUNK_RUN_USER="archStudent"

# Populate this file with a list of hosts that this script should install to,
# with one host per line. This must be specified in the form that should
# be used for the ssh login, ie. username@host
#
# Example file contents:
# splunkuser@10.20.13.4
# splunkker@10.20.13.5
  HOSTS_FILE="uf_hosts"

# This should be a WGET command that was *carefully* copied from splunk.com!!
# Sign into splunk.com and go to the download page, then look for the wget
# link near the top of the page (once you have selected your platform)
# copy and paste your wget command between the ""
  WGET_CMD="wget -O splunkforwarder-7.1.2-a0c72a66db66-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.1.2&product=universalforwarder&filename=splunkforwarder-7.1.2-a0c72a66db66-Linux-x86_64.tgz&wget=true'"
# Set the install file name to the name of the file that wget downloads
# (the second argument to wget)
  INSTALL_FILE="splunkforwarder-7.1.2-a0c72a66db66-Linux-x86_64.tgz"

# After installation, the forwarder will become a deployment client of this
# host.  Specify the host and management (not web) port of the deployment server
# that will be managing these forwarder instances.
# Example 1.2.3.4:8089
  DEPLOY_SERVER="x.x.x.x:8089"

# Set the seed app folder name for deploymentclien.conf
  DEPLOY_APP_FOLDER_NAME="seed_all_deploymentclient"
# Set the new Splunk admin password
  PASSWORD="changeme"

REMOTE_SCRIPT_DEPLOY="
  cd /opt
  sudo $WGET_CMD
  sudo tar xvzf $INSTALL_FILE
  sudo rm $INSTALL_FILE
  sudo useradd $SPLUNK_RUN_USER
  sudo chown -R $SPLUNK_RUN_USER:$SPLUNK_RUN_USER /opt/splunkforwarder
  echo \"[user_info]
USERNAME = admin
PASSWORD = $PASSWORD\" > /opt/splunkforwarder/etc/system/local/user-seed.conf  
  mkdir -p /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local
  echo \"[target-broker:deploymentServer]
targetUri = $DEPLOY_SERVER\" > /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local/deploymentclient.conf
  sudo -u $SPLUNK_RUN_USER /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt
  sudo /opt/splunkforwarder/bin/splunk enable boot-start -user $SPLUNK_RUN_USER

  exit
 "

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"

#===============================================================================================
  echo "In 5 seconds, will run the following script on each remote host:"
  echo
  echo "===================="
  echo "$REMOTE_SCRIPT_DEPLOY"
  echo "===================="
  echo 
  sleep 5
  echo "Reading host logins from $HOSTS_FILE"
  echo
  echo "Starting."
  for DST in `cat "$DIR/$HOSTS_FILE"`; do
    if [ -z "$DST" ]; then
      continue;
    fi
    echo "---------------------------"
    echo "Installing to $DST"
    echo "Initial UF deployment"
    sudo ssh -t "$DST" "$REMOTE_SCRIPT_DEPLOY"
  done  
  echo "---------------------------"
  echo "Done"
  echo "Please use the following app folder name to override deploymentclient.conf options: $DEPLOY_APP_FOLDER_NAME"

[damode]

I got the below error, when I ran this script,

    bash: line 7: /opt/splunkforwarder/etc/system/local/user-seed.conf: Permission denied
    bash: line 9: /opt/splunkforwarder/etc/apps/org_all_deploymentclient/local/deploymentclient.conf: Permission denied

[serjandrosov]

Thanks for the highlight, updated the script with moving chown command earlier than echo

[jos_kaats]

You could tweak the script so that you do the chown cmd before you write to the user-seed.conf and deploymentclient.conf. And then prefix the file creating echo cmds with sudo -u $SPLUNK_RUN_USER .
After chown, the splunk user should have write permissions in the splunk directories.

[guarisma]

@damode the error is pretty clear, you have a permission issue, please check that the user account running the script has the sufficient permissions to read and/or write those files.
Good Luck!

[damode]

I tried adding sudo before
`echo \"[user_info]
USERNAME = admin
PASSWORD = $PASSWORD\" > /opt/splunkforwarder/etc/system/local/user-seed.conf

mkdir -p /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local

echo \"[target-broker:deploymentServer]
targetUri = $DEPLOY_SERVER\" > /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local/deploymentclient.conf` commands, but still didnt make any difference. All other commands worked without any issues.

[guarisma]

for this case, sudo is only used for the echo command, once you get to the redirection '>', you are back to your user

[a_salikov]

Hello everyone,
Can you help me, when I copy and past script in notepad++ and after changing and launch script on server I have different errors:

scriptUF.sh: line 2: $'\r': command not found
scriptUF.sh: line 8: $'\r': command not found
scriptUF.sh: line 22: $'\r': command not found
scriptUF.sh: line 23: $'\r': command not found
In 5 seconds, will run the following script on each remote host:

scriptUF.sh: line 25: $'echo\r': command not found

====================
scriptUF.sh: line 29: $'echo\r': command not found
sleep: invalid time interval ‘5\r’
Try 'sleep --help' for more information.
Reading host logins from /opt/forwarderlist
scriptUF.sh: line 32: $'echo\r': command not found
Starting.
scriptUF.sh: line 34: syntax error near unexpected token $'do\r''
'criptUF.sh: line 34:for DST in cat "$HOSTS_FILE"; do

Please, help me
Thank you very much.

[sarudhra]

If you have created the script file in windows you would get this error while running from Linux . Try creating the script file in Linux vi editor, paste the script and save .

Alternatively you could run the following command for your script file to remove trailing \r character 

sed -i 's/\r$//' filename

[princemanto2580]

Can I get similar kind of thing for Windows Universal Forwarder remote installation. I mean, like very straight forward solution.

[MartinMcNutt]

Windows Comments

I work in a rather large corporation and the politics/ paperwork usually prevent me from going the official route. It is just easier to tackle the installs myself using Old school utilities that most people forget about.

For me, all I do is prep the server by coping down the binaries to the server. This directories contains the install.cmd file with the command line options you want to include.

xcopy /e/s /i \\<sourceServer>\<sharename>\SplunkUF\*.* \\<destserver>\<Drive>$\Software\SplunkUF

Once the server is prepped all I do is run psexec from Sysinterals (Microsoft Utility)

psexec \\<servername> <local path to file>

Detailed example

xcopy /e/s /i \\Splunk\Software\SplunkUF\*.* \\ServerA\D$\Software\SplunkUF 
psexec \\servera D:\software\SplunkUF\Install.cmd

You will have to play around with the options. Adjust to your liking! Example PSexec will wait until the install is finished before moving on. If you simply want to have 20 servers install at once put a -d that will tell psexec not to wait.

Now if you use a deployment server to manage the configuration. You can specify the server in the install.cmd and life is good.

However, you should consider taking a different approach with the Deployment server and make an app for the deployment server and copy down that directory to etc/apps folder. Maybe include that as part of your simple install.cmd.

When I did my testing, i found that it put the deployment server in etc/system/local which hard coded it and I wont be able to change the server via deployment server. When it is in the apps folder I can just send out another app to update the location if I need to move it to another server. Example if I had to move from a VM to a physical server.

Just my 2 Cents

PS NOTEPAD++ is a must with splunk 🙂

[lguinn2]

Thanks Martin, for adding to this discussion!

[lguinn2]

BTW, if you want to learn more about the Deployment Server and how to distribute configuration files (like inputs.conf), take a look at the Distributed Deployment Manual, especially here:

5.x http://docs.splunk.com/Documentation/Splunk/5.0.2/Deploy/Aboutdeploymentserver

6.0 http://docs.splunk.com/Documentation/Splunk/6.0/Updating/Aboutdeploymentserver

[lguinn2 · Accepted solution]

With some updates that were created (and tested!) by rroberts. Thanks Mr R; you are the best!
If you cut-and-paste this script - watch out for the line wraps! You will need to correct these before your script will run.

#!/bin/sh

# This EXAMPLE script shows how to deploy the Splunk universal forwarder
# to many remote hosts via ssh and common Unix commands.
# For "real" use, this script needs ERROR DETECTION AND LOGGING!!

# --Variables that you must set -----

# Populate this file with a list of hosts that this script should install to,
# with one host per line. This must be specified in the form that should
# be used for the ssh login, ie. username@host
#
# Example file contents:
# splunkuser@10.20.13.4
# splunkker@10.20.13.5
HOSTS_FILE="$HOME/whereToInstallUF"

# This should be a WGET command that was *carefully* copied from splunk.com!!
# Sign into splunk.com and go to the download page, then look for the wget
# link near the top of the page (once you have selected your platform)
# copy and paste your wget command between the ""
WGET_CMD="wget -O splunkforwarder..."

# Set the install file name to the name of the file that wget downloads
# (the second argument to wget)
INSTALL_FILE="splunkforwarder..."

# After installation, the forwarder will become a deployment client of this
# host.  Specify the host and management (not web) port of the deployment server
# that will be managing these forwarder instances.
DEPLOY_SERVER="xxx.xxx.xxx.xxx:8089"

# Set the new Splunk admin password
PASSWORD="newpassword"

# ----------- End of user settings -----------

# create script to run remotely. Watch out for line wraps, esp. in the "set deploy-poll" line below.  
# the remote script assumes that 'splunkuser' (the login account) has permissions to write to the
# /opt directory (this is not generally the default in Linux)
REMOTE_SCRIPT="
cd /opt
$WGET_CMD
tar -xzf $INSTALL_FILE
# /opt/splunkforwarder/bin/splunk enable boot-start -user splunkusername
/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt
/opt/splunkforwarder/bin/splunk set deploy-poll \"$DEPLOY_SERVER\" --accept-license --answer-yes --auto-ports --no-prompt  -auth admin:changeme
/opt/splunkforwarder/bin/splunk edit user admin -password $PASSWORD -auth admin:changeme
/opt/splunkforwarder/bin/splunk restart
"    
echo "In 5 seconds, will run the following script on each remote host:"
echo
echo "===================="
echo "$REMOTE_SCRIPT"
echo "===================="
echo
sleep 5
echo "Reading host logins from $HOSTS_FILE"
echo
echo "Starting."

for DST in `cat "$HOSTS_FILE"`; do
  if [ -z "$DST" ]; then
    continue;
  fi
  echo "---------------------------"
  echo "Installing to $DST"

  # run script on remote host - you will be prompted for the password
  ssh "$DST" "$REMOTE_SCRIPT"

done  
echo "---------------------------"
echo "Done"

[payal4296]

How UF will be installed on newer version of Splunk with no default password "changeme"??

[kuja]

Anyone else have an issue with the deployment server not being set after the deployment on the forwarder?

[SONY_anilyelmar]

Permissions issue can be resolved using your login non-root  user . Sequence like below: my non-root user which use wget is  sccStudent:sccStudent

REMOTE_SCRIPT_DEPLOY="
   cd /opt
   sudo /opt/splunkforwarder/bin/splunk stop
   sudo rm -rf /opt/splunkforwarder
   sudo $WGET_CMD
   #sudo chmod 755 splunkforwarder-8.2.3-cd0848707637-Linux-x86_64.tgz
   sudo tar xvzf $INSTALL_FILE
   sudo rm $INSTALL_FILE
   sudo mkdir -p /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local
   sudo chown -R sccStudent:sccStudent /opt/splunkforwarder
   sudo  echo \"[user_info]
USERNAME = admin
PASSWORD = $PASSWORD\" > /opt/splunkforwarder/etc/system/local/user-seed.conf  
   sudo echo \"[target-broker:deploymentServer]
targetUri = $DEPLOY_SERVER\" > /opt/splunkforwarder/etc/apps/$DEPLOY_APP_FOLDER_NAME/local/deploymentclient.conf
   sudo useradd $SPLUNK_RUN_USER
   sudo chown -R $SPLUNK_RUN_USER:$SPLUNK_RUN_USER /opt/splunkforwarder
   sudo -u $SPLUNK_RUN_USER /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt
   sudo /opt/splunkforwarder/bin/splunk enable boot-start -user $SPLUNK_RUN_USER
  exit
 "

[mdsnmss]

I ran into as well. The issue is on this command:

sudo -u splunk /opt/splunkforwarder/bin/splunk set deploy-poll \"$DEPLOY_SERVER\" --accept-license --answer-yes --auto-ports --no-prompt  -auth admin:changeme

I changed it to:

sudo -u splunk /opt/splunkforwarder/bin/splunk set deploy-poll $DEPLOY_SERVER -auth admin:changeme

There were some unneeded bits as part of the command.

[mansel_scheffel]

I am.. did you ever find out what the issue was?