Why are Splunk Forwarders "re-configuring" every 10 minutes according to event logs?

[jmaple]

We noticed while investigating issues that the Splunk Forwarder is repeatedly "re-configuring" itself using the MSI package? Here is the event we keep seeing.

03/08/2016 01:26:15 PM
LogName=Application
SourceName=MsiInstaller
EventCode=1035
EventType=4
Type=Information
ComputerName=hostname
User=NOT_TRANSLATED
Sid=S-1-5-18
SidType=0
TaskCategory=None
OpCode=Info
RecordNumber=61598476
Keywords=Classic
Message=Windows Installer reconfigured the product. Product Name: UniversalForwarder. Product Version: 6.3.1.0. Product Language: 1033. Manufacturer: Splunk, Inc.. Reconfiguration success or error status: 0.

This event happens 12 times every 10 minutes. Has anyone else seen this happening?

[Richfez · Accepted solution]

It isn't necessarily related to Splunk directly though it probably is.

I just had the same issue with a SQL box "reconfiguring" SQL because it had been patched but not yet rebooted. It was a fight between the updated version and the unupdated version, and a reboot took care of it. I've seen it happen when the installer needs to swap out files but has a service or lock it can't stop or fix.

Here's a big long and only partially applicable look at some of the things to check if a reboot doesn't resolve this.

[muebel]

Are you running any scripted inputs, or any inputs at all that interact with wmi? In particular win32_product?

[jmaple]

Currently we only use that which is included in the "Splunk_TA_windows" app and we don't have all inputs that are stock with the app enabled. I don't believe we have any inputs that interact with WMI that repeats at 10 minute intervals.