How can I automate the downloading of universal forwarder?

[petersonjared]

Everything I am reading is that to download via wget, cURL, etc, that you have to specify the full path that contains the specific version number in the name/path. How can I get the latest/current version through automation versus hard-coding the path?

[whrg]

Hello @petersonjared,

You can see the latest download links in the HTML source code of https://www.splunk.com/en_us/download/universal-forwarder.html. Same for Splunk Enterprise with https://www.splunk.com/en_us/download/splunk-enterprise.html. Search for "data-link" and you will find the relevant lines which look like this:

<a class="splunk-btn sp-btn-solid sp-btn-pink" data-arch="x86_64" data-filename="splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" data-link="https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi" ...

I use grep to extract all such data-links:

$ curl https://www.splunk.com/en_us/download/universal-forwarder.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/windows/splunkforwarder-9.1.2-b6b9c8185839-x86-release.msi
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.deb
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.aarch64.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-armv8.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-ppc64le.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.ppc64le.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839-Linux-s390x.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/linux/splunkforwarder-9.1.2-b6b9c8185839.s390x.rpm
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-universal2.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/osx/splunkforwarder-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-FreeBSD11-amd64.tgz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/freebsd/splunkforwarder-9.1.2-b6b9c8185839-freebsd-11.3-amd64.txz
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-x86_64.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-intel.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-solaris-sparc.p5p
https://download.splunk.com/products/universalforwarder/releases/9.1.2/solaris/splunkforwarder-9.1.2-b6b9c8185839-SunOS-sparc.tar.Z
https://download.splunk.com/products/universalforwarder/releases/9.1.2/aix/splunkforwarder-9.1.2-b6b9c8185839-AIX-powerpc.tgz

Same for Splunk Enterprise:

$ curl https://www.splunk.com/en_us/download/splunk-enterprise.html | grep -o -P 'data-link="[^"]+' | cut -c 12-
https://download.splunk.com/products/splunk/releases/9.1.2/windows/splunk-9.1.2-b6b9c8185839-x64-release.msi
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-Linux-x86_64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839.x86_64.rpm
https://download.splunk.com/products/splunk/releases/9.1.2/linux/splunk-9.1.2-b6b9c8185839-linux-2.6-amd64.deb
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-darwin-64.tgz
https://download.splunk.com/products/splunk/releases/9.1.2/osx/splunk-9.1.2-b6b9c8185839-macosx-10.11-intel.dmg

[evolart]

Awesome solution. Now I just need to figure out how I could achieve the same in Powershell since we automate installation through AWS SSM.

[ReverendRhyme]

Did you ever figure out how to automate the download in PowerShell? Looking for something similar.

[kenzo1234]

as I hit download from https://www.splunk.com/en_us/download/universal-forwarder.html#tabs/freebsd

it will only send me to the splunk software license agreement page and not going anywhere else. please tell me how can I download the file.

[isoutamo]

Hi

here is how we have done this. https://community.splunk.com/t5/Installation/installing-splunk-forwarder-using-ansible/m-p/511021

r. Ismo