How to ensure logs generated during Universal Forwarder upgrade are not lost or duplicated?

[cboillot]

We are about to upgrade several hundred Universal Forwarders (UF) in our environment. We want to make sure that any logs that were generated during the upgrade of the UF would not be lost or duplicated. I did find info on current_only, however it seem this is only for the Windows Event Log Monitor, and not the MONITOR:.

Is there anything we need to make sure we have in place?

How will the UF know where the old version left off?

I have tried to look this up, but with all the posts just named Universal Forwarder, I could have overlooked if this has been asked before.

[gjanders · Accepted solution]

What you are referring to is inbuilt Splunk functionality, as per Monitor files and directories

When the Splunk server is restarted,
it continues processing files where it
left off. It first checks for the file
or directory specified in a monitor
configuration. If the file or
directory is not present on start,
Splunk Enterprise checks for it every
24 hours from the time of the last
restart. The monitor process scans
subdirectories of monitored
directories continuously.

Effectively Splunk keeps a checkpoint of where it got to in a file in the fishbucket (an internal filestore within the forwarder), so unless your wiping out the Splunk installation directory an upgrade will not cause any issues as the files will not be deleted by upgrades.

You do have some controls around determining if the file has been seen before in the inputs.conf , in particular refer to initCrcLength . You only need to adjust this if you see issues in the splunkd.log from the forwarder, by default this functionality should just work!

[cboillot]

Thank you! This is what I thought, but was asked to get verification.

[gjanders]

Not a problem, you can send feedback to the documentation team if it is not clear enough, they are usually happy to take feedback...