Splunk Add-on for Microsoft Windows: Why is timestamp extraction on Windows Event Logs failing?

[karlbosanquet]

I have a clustered environment with Splunk Add-on for Microsoft Windows deployed to Indexers, Search Heads and Universal Forwarders.

I have an additional application deployed to Indexers and Search Heads to handle the Logbinder WinEventLog. The contents of $SPLUNK_HOME/etc/slave_apps/logbinder/local/props.conf is;

[WinEventLog:LOGbndSP]
TIME_PREFIX = Occurred:\s
TIME_FORMAT = %Y/%m/%d %H:%M:%S

I have also created a $SPLUNK_HOME/etc/slave_apps/logbinder/metadata/meta.local with the following;

[]
export=system

The format of the log is the standard WinEventLog format with the following in the Message details;

Occurred: 2016/12/01 19:00:59

I have ran btool on the Indexers, where the time parsing happens, and the the results show that the TIME_PREFIX and TIME_FORMAT are being picked up, however the value being populated into _time is the Windows Event time, not the time specified starting with Occurred.

[karlbosanquet · Accepted solution]

After discussing this with Splunk Professional Services, it has been highlighted that not all time extraction happens at the indexer.

WinMon is baked into Splunk and so the time extraction for [WinEventLog:...] is assigned as early as the Universal Forwarder, and CANNOT be changed.

[woodcock]

There is no way to do this. You cannot even overwrite the metadata after the fact. You will either need Splunk to add a feature OR you will need to completely reproduce (clone and modify) the existing modular input yourself.

[karlbosanquet]

Still unable to resolve this. The data is being collected by a Universal Forwarder with the Splunk Windows TA installed, could this be causing issues?

The TA is also installed on indexers and search heads, however as stated the logbinder app is unable to change the _time value.

I also ran btool on the indexers and the results show it should be using the values from the logbinder app for this sourcetype.

[somesoni2]

The timeformat should be %Y/%m/%d %H:%M:%S instead of Y%/%m/%d %H:%M:%S. Also, you don't have any Heavy Forwarder in front of indexer right? Also, the Windows Event log that you collect from the servers, does it have Universal Forwarder installed or Splunk Enterprise version?

[karlbosanquet]

The format in the original question was a misprint by me, the app has the correct format as you stated.

No heavy forwarders, just Universal Forwarder direct to Index Cluster Master, then to Indexers.