topic Re: Simple installation script for Universal Forwarder in Getting Data In

Simple installation script for Universal Forwarder

lguinn2 — Tue, 22 Nov 2011 19:08:02 GMT

When you have more than a few forwarders to maintain, it becomes tedious (and error-prone) to install them one-by-one. Using the Deployment Server is great for keeping the configurations up-to-date and consistent.

But I want to script the installation. This isn't really a Splunk problem, it's more of a scripting problem.

Any great scripts out there? All answers welcome, especially Linux and Windows.

Re: Simple installation script for Universal Forwarder

lguinn2 — Tue, 22 Nov 2011 19:09:42 GMT

With some updates that were created (and tested!) by rroberts. Thanks Mr R; you are the best!
If you cut-and-paste this script - watch out for the line wraps! You will need to correct these before your script will run.

#!/bin/sh

# This EXAMPLE script shows how to deploy the Splunk universal forwarder
# to many remote hosts via ssh and common Unix commands.
# For "real" use, this script needs ERROR DETECTION AND LOGGING!!

# --Variables that you must set -----

# Populate this file with a list of hosts that this script should install to,
# with one host per line. This must be specified in the form that should
# be used for the ssh login, ie. username@host
#
# Example file contents:
# splunkuser@10.20.13.4
# splunkker@10.20.13.5
HOSTS_FILE="$HOME/whereToInstallUF"

# This should be a WGET command that was *carefully* copied from splunk.com!!
# Sign into splunk.com and go to the download page, then look for the wget
# link near the top of the page (once you have selected your platform)
# copy and paste your wget command between the ""
WGET_CMD="wget -O splunkforwarder..."

# Set the install file name to the name of the file that wget downloads
# (the second argument to wget)
INSTALL_FILE="splunkforwarder..."

# After installation, the forwarder will become a deployment client of this
# host.  Specify the host and management (not web) port of the deployment server
# that will be managing these forwarder instances.
DEPLOY_SERVER="xxx.xxx.xxx.xxx:8089"

# Set the new Splunk admin password
PASSWORD="newpassword"

# ----------- End of user settings -----------

# create script to run remotely. Watch out for line wraps, esp. in the "set deploy-poll" line below.  
# the remote script assumes that 'splunkuser' (the login account) has permissions to write to the
# /opt directory (this is not generally the default in Linux)
REMOTE_SCRIPT="
cd /opt
$WGET_CMD
tar -xzf $INSTALL_FILE
# /opt/splunkforwarder/bin/splunk enable boot-start -user splunkusername
/opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt
/opt/splunkforwarder/bin/splunk set deploy-poll \"$DEPLOY_SERVER\" --accept-license --answer-yes --auto-ports --no-prompt  -auth admin:changeme
/opt/splunkforwarder/bin/splunk edit user admin -password $PASSWORD -auth admin:changeme
/opt/splunkforwarder/bin/splunk restart
"    
echo "In 5 seconds, will run the following script on each remote host:"
echo
echo "===================="
echo "$REMOTE_SCRIPT"
echo "===================="
echo
sleep 5
echo "Reading host logins from $HOSTS_FILE"
echo
echo "Starting."

for DST in `cat "$HOSTS_FILE"`; do
  if [ -z "$DST" ]; then
    continue;
  fi
  echo "---------------------------"
  echo "Installing to $DST"

  # run script on remote host - you will be prompted for the password
  ssh "$DST" "$REMOTE_SCRIPT"

done  
echo "---------------------------"
echo "Done"

Re: Simple installation script for Universal Forwarder

lguinn2 — Tue, 22 Nov 2011 19:10:31 GMT

This was the shortest script that I could come up with that wasn't completely lame. It is based on a much more sophisticated script that is now part of the documentation.

Re: Simple installation script for Universal Forwarder

lguinn2 — Tue, 22 Nov 2011 19:12:57 GMT

If you cut-and-paste this script, watch out for the line wrap on the lines that start with
/opt/splunkforwarder

Re: Simple installation script for Universal Forwarder

lguinn2 — Sun, 27 Nov 2011 20:49:07 GMT

Documentation here, including another example script...

Re: Simple installation script for Universal Forwarder

ssbarnea — Wed, 21 Mar 2012 18:32:07 GMT

Thanks, the above script is a proof of the splunk inability to create easy to install and deploy products. I just hope that they will fix the current silly URL download tricks and provide a solution like this, maybe even RPM and DEB repositories.

Re: Simple installation script for Universal Forwarder

bilhays — Fri, 30 Mar 2012 14:42:16 GMT

ssbarnea, that's a bit harsh. We're talking about on the fly configuration, which is not trivial, and this kind of script is very helpful (at least to oldschoolers like myself). Thanks lguinn, this one answers 3-4 questions I had while writing something similar.

Re: Simple installation script for Universal Forwarder

dwaddle — Thu, 14 Jun 2012 02:59:42 GMT

A couple of comments:

[1] Nothing keeps you from running a purely-internal yum repository to distribute Splunk install images to your servers. Then, "yum update" even works...
[2] Rather than setting the admin password directly, there's always user-seed.conf, which works well for that purpose.
[3] It's not terribly difficult to spin out your own RPM that includes splunk as a dependency - let that RPM (and its included scriplets) deal with post-install configuration

Re: Simple installation script for Universal Forwarder

rmorlen — Thu, 10 Jan 2013 11:48:47 GMT

Thanks for the script.

We have found Splunk incredibly easy to install and upgrade. Much easier that most products. We have UF's installed on 3000+ servers.

Re: Simple installation script for Universal Forwarder

lguinn2 — Mon, 15 Apr 2013 17:56:35 GMT

BTW, if you want to learn more about the Deployment Server and how to distribute configuration files (like inputs.conf), take a look at the Distributed Deployment Manual, especially here:

5.x http://docs.splunk.com/Documentation/Splunk/5.0.2/Deploy/Aboutdeploymentserver

6.0 http://docs.splunk.com/Documentation/Splunk/6.0/Updating/Aboutdeploymentserver

Re: Simple installation script for Universal Forwarder

jviteka — Wed, 16 Oct 2013 22:31:03 GMT

Do I need to create a share for this?

Re: Simple installation script for Universal Forwarder

MartinMcNutt — Wed, 16 Oct 2013 23:38:37 GMT

Windows Comments

I work in a rather large corporation and the politics/ paperwork usually prevent me from going the official route. It is just easier to tackle the installs myself using Old school utilities that most people forget about.

For me, all I do is prep the server by coping down the binaries to the server. This directories contains the install.cmd file with the command line options you want to include.

xcopy /e/s /i \\<sourceServer>\<sharename>\SplunkUF\*.* \\<destserver>\<Drive>$\Software\SplunkUF

Once the server is prepped all I do is run psexec from Sysinterals (Microsoft Utility)

psexec \\<servername> <local path to file>

Detailed example

xcopy /e/s /i \\Splunk\Software\SplunkUF\*.* \\ServerA\D$\Software\SplunkUF 
psexec \\servera D:\software\SplunkUF\Install.cmd

You will have to play around with the options. Adjust to your liking! Example PSexec will wait until the install is finished before moving on. If you simply want to have 20 servers install at once put a -d that will tell psexec not to wait.

Now if you use a deployment server to manage the configuration. You can specify the server in the install.cmd and life is good.

However, you should consider taking a different approach with the Deployment server and make an app for the deployment server and copy down that directory to etc/apps folder. Maybe include that as part of your simple install.cmd.

When I did my testing, i found that it put the deployment server in etc/system/local which hard coded it and I wont be able to change the server via deployment server. When it is in the apps folder I can just send out another app to update the location if I need to move it to another server. Example if I had to move from a VM to a physical server.

Just my 2 Cents

PS NOTEPAD++ is a must with splunk 🙂

Re: Simple installation script for Universal Forwarder

lguinn2 — Sun, 20 Oct 2013 03:26:59 GMT

Thanks Martin, for adding to this discussion!

Re: Simple installation script for Universal Forwarder

lguinn2 — Wed, 21 May 2014 23:59:19 GMT

Another script posting - also good

Forwarder Installation Script

Re: Simple installation script for Universal Forwarder

rwells — Fri, 04 Sep 2015 18:21:18 GMT

What does DST mean in this scrip and what does it do?

Re: Simple installation script for Universal Forwarder

lguinn2 — Fri, 04 Sep 2015 20:44:18 GMT

DST is a variable. On each iteration of the loop, DST is set to the next host in the HOSTS_FILE.
So at any given point, DST should contain the IP address where the universal forwarder is to be installed.

Re: Simple installation script for Universal Forwarder

tv5 — Thu, 19 Nov 2015 19:33:54 GMT

Thanks Iguinn, tweaked this a bit for non-root user account... and this works!

 #!/bin/sh

 HOSTS_FILE="/opt/forwarderlist"
 WGET_CMD="wget -O splunkforwarder..."
 INSTALL_FILE="splunkforwarder-6....tgz"
 DEPLOY_SERVER="xx.xx.xx.xx:8089"
 PASSWORD="newpassword"

 REMOTE_SCRIPT="
 cd /opt
 sudo $WGET_CMD
 sudo tar -xzf $INSTALL_FILE

 sudo useradd -m -r splunk
 sudo chown -R splunk:splunk /opt/splunkforwarder

 ### /opt/splunkforwarder/bin/splunk enable boot-start -user splunk
 sudo -u splunk /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt
 sudo -u splunk /opt/splunkforwarder/bin/splunk set deploy-poll \"$DEPLOY_SERVER\" --accept-license --answer-yes --auto-ports --no-prompt  -auth admin:changeme
 sudo -u splunk /opt/splunkforwarder/bin/splunk edit user admin -password $PASSWORD -auth admin:changeme
 sudo -u splunk /opt/splunkforwarder/bin/splunk restart
 "    

 echo "In 5 seconds, will run the following script on each remote host:"
 echo
 echo "===================="
 echo "$REMOTE_SCRIPT"
 echo "===================="
 echo
 sleep 5
 echo "Reading host logins from $HOSTS_FILE"
 echo
 echo "Starting."
 for DST in `cat "$HOSTS_FILE"`; do
   if [ -z "$DST" ]; then
     continue;
   fi
   echo "---------------------------"
   echo "Installing to $DST"
   sudo ssh -t "$DST" "$REMOTE_SCRIPT"
 done  
 echo "---------------------------"
 echo "Done"

Re: Simple installation script for Universal Forwarder

kuja — Thu, 04 Feb 2016 22:25:36 GMT

Anyone else have an issue with the deployment server not being set after the deployment on the forwarder?

Re: Simple installation script for Universal Forwarder

mansel_scheffel — Sun, 14 Aug 2016 14:31:51 GMT

I am.. did you ever find out what the issue was?

Re: Simple installation script for Universal Forwarder

princemanto2580 — Sun, 19 Feb 2017 06:40:11 GMT

Can I get similar kind of thing for Windows Universal Forwarder remote installation. I mean, like very straight forward solution.