Upgrading a Universal Forwarder

[itsmevic]

Hello, we have around 1200 systems that have UF's on them.  They are a mixture of both Windows and Linux devices.  I'm curious if it's possible to use a platform like Tanium or SCCM to push the UF .MSI down to those systems to initiate the upgrade that way? Your input is GREATLY appreciated.  Thanks.

[isoutamo]

Hi

definitely you could and actually should use some automation/ management tool to updating etc UFs. I think that there haven’t been any specific requirements for UFs. You could manage those like any other similar programs. 
Then it’s up to you if you want to use the same tool for managing configurations or do you use DS for it. Both works and selection depends on you environment and processes   
r. Ismo

[SamHTexas]

Hello & thank u for your message. We are going to follow your earlier recommendation using Ansible & SCCM to upgrade the Win & Linux UFs & HFs. We have different versions or UFs to upgrade to 8.2.2. any issues or order to upgrade them? Also should the UFs /HFs be upgraded before the Splunk servers (Instances) or after please? Thanks very much.

[isoutamo]

Hi

Splunk's main guideline is that Splunk servers should be on latest level 1st and then UFs and HFs should be less or same level than servers. 

Update order of servers and apps:

• https://community.splunk.com/t5/Installation/What-s-the-order-of-operations-for-upgrading-Splunk-Ent...

Compatibility matrix:

• https://docs.splunk.com/Documentation/VersionCompatibility/current/Matrix/CompatMatrix
• https://docs.splunk.com/Documentation/VersionCompatibility/current/Matrix/Compatibilitybetweenforwar...

There shouldn't be any issues when upgrading UFs or HFs. Only thing which come my mind is, just update those not remove + install! And of course check first all apps in HFs that those are compatible with the new version (e.g. python, java etc.) especially if you do a main updates like 6.x/7.x to 8.x.

r. Ismo

[richgalloway · Accepted solution]

Not only is it possible, it's recommended for large installations.  Many customers use such tools to install and upgrade forwarders.  See the UF manual at https://docs.splunk.com/Documentation/Forwarder/8.2.2/Forwarder/InstallaWindowsuniversalforwarderrem... for more information.

[PickleRick]

As a side question - how do you keep track of certificate validity in such big environment? Or do you just use unencrypted connections?

[isoutamo]

At least the next two solutions come into my mind:

1. Use same cert on all UFs with same expiration days etc. There could be some differences between linux/windows (cannot check it now)
   1. One for unix/linux
   2. Another in Windows
2. Keep individual certificates on some repository/sw and check automatically those expiration days and reinstall when needed.

In most cases I think that option 1 is enough to get needed security level vs. time/money which are needed to implement second one.

[PickleRick]

I'm asking because I have enough hassle with "cert accounting" with around 50 UF's.

But I use mutual auth with a list of permitted peers so it's necessary that each component has its own valid cert.

And I must admit it's not easy to manage certs for windows-based UF's. Especially that there are many components that are not within a single domain and are not centrally managed. If I had WinRM enabled, I could use ansible. But I don't have it.

Of course one could do that in the form of an app pushed from the deployment server (the certfile path can point to a directory within an app contents) but it's extremely ugly from both the security point of view - you'd have to push the private keys from the deployment server, and the manageability - you'd have to have a different app for each UF.