We want XML based logs over Non-XML logs, but we are seeing both for some reason. Moreover, if we look at the log messages with source=WinEventLog:Security for example, the sourcetype shows 'xmlwineventlog'. Is this normal/expected behavior or is there some additional tuning we need to do?
Turns out the cause of this was the windows TA add-on was not installed on all our indexers. This now parses the log "source" as the XML name consistently. It was not duplicating logs.
Thank you all for the support.
Check inputs.conf, if you have more than where one says renderXml=true, another says renderXml=false.
can you also check if you are seeing xml and non-xml events for a same host?
There is no 'renderXml=false' in our inputs.conf, only 'renderXml=true'
No we do not see duplicate events logs for both a single host. What's odd is, we do not see the same log record ID's for 'source = XmlWinEventLog:Security' versus 'source = WinEventLog:Security'. But we are getting logs for both sources...
You have confirmed that you are not seeing xml and non-xml from same host.
can you verify inputs.conf pushed to client which is sending xml events and also verify inputs.conf pushed to client which is sending non-xml events?
I am sure there would be a difference between the inputs used in two different servers.
Can you check if you maybe have "Remote event log collections" enabled for this host on one of your Splunk instances? This is one of the reasons why there can be duplicate data..
Looking at the wmi.conf file on one of our universal forwarders, I see remote log collection is disabled:
[settings]
initial_backoff = 5
max_backoff = 20
max_retries_at_max_backoff = 0
checkpoint_sync_interval = 2
## Pull event logs FROM the local system
## Usually disabled in favor of using WinEventLog inputs
[WMI:LocalApplication]
interval = 10
event_log_file = Application
disabled = 1
[WMI:LocalSystem]
interval = 10
event_log_file = System
disabled = 1
[WMI:LocalSecurity]
interval = 10
event_log_file = Security
disabled = 1
Turns out the cause of this was the windows TA add-on was not installed on all our indexers. This now parses the log "source" as the XML name consistently. It was not duplicating logs.
Thank you all for the support.
Faced same issue, it was because we sent logs before installing Splunk Add-on For MS Windows on Indexer. Before this Add-on logs were tagged with source=WinEventLogs and after installing this Add-on the logs are tagged with source=xmlWinEventLogs. No duplicate events, just change in source tagging.