Getting Data In

How can I automate the downloading of universal forwarder?

petersonjared
Explorer

Everything I am reading is that to download via wget, cURL, etc, that you have to specify the full path that contains the specific version number in the name/path. How can I get the latest/current version through automation versus hard-coding the path?

whrg
Motivator

Hello @petersonjared,

You can see the latest download links in the HTML source code of https://www.splunk.com/en_us/download/universal-forwarder.html. Same for Splunk Enterprise with https://www.splunk.com/en_us/download/splunk-enterprise.html. Search for "data-link" and you will find the relevant lines which look like this:

<a class="splunk-btn sp-btn-solid sp-btn-pink" data-arch="x86_64" data-filename="splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" data-link="https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" ...

I use grep to extract all such data-links:

$ curl https://www.splunk.com/en_us/download/universal-forwarder.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x86-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.aarch64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-ppc64le.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.ppc64le.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-s390x.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.s390x.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-FreeBSD11-amd64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-freebsd-11.3-amd64.txz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-x86_64.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-intel.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-sparc.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-sparc.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/aix/splunkforwarder-9.1.2-b6b9c8185839-AIX-powerpc.tgz

Same for Splunk Enterprise:

$ curl https://www.splunk.com/en_us/download/splunk-enterprise.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/splunk/releases/9.1.2/windows/splunk-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg

evolart
Engager

Awesome solution. Now I just need to figure out how I could achieve the same in Powershell since we automate installation through AWS SSM.

0 Karma

ReverendRhyme
Engager

Did you ever figure out how to automate the download in PowerShell? Looking for something similar.

0 Karma

kenzo1234
Observer

as I hit download from https://www.splunk.com/en_us/download/universal-forwarder.html#tabs/freebsd

it will only send me to the splunk software license agreement page and not going anywhere else. please tell me how can I download the file.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...