Getting Data In

Why are date_* fields are not being extracted from Windows security logs?

wkupersa
Path Finder

We are bringing Windows Security Logs into Splunk via the universal forwarder. All of the events begin with a timestamp. It is evident in the raw event. However none of the `date*` fields populate. I am assuming this must be a common issue since we aren't doing anything special and these are just Windows Events, but I don't see this question posted here already!?

Do we need to do something special to get the timestamp to parse and get the date_* fields to populate on ingestion?

Thanks!

1 Solution

dwaddle
SplunkTrust
SplunkTrust

The date_* fields come from parsing of timestamps. They are basically a useful side-effect. If the timestamp is not parsed, the date_* fields will not appear. Windows event logs come in via a modular input these days. The modular input sends in the pre-parsed time as it comes from the Windows event log APIs. So Splunk does not have to do any timestamp parsing, and therefore we don't get the date_* fields.

Cross-links to other similar questions:

http://answers.splunk.com/answers/30822/date-hour-not-present-in-wineventlogs.html
http://answers.splunk.com/answers/92087/default-fields-are-not-visible.html#comment-92199

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

The date_* fields come from parsing of timestamps. They are basically a useful side-effect. If the timestamp is not parsed, the date_* fields will not appear. Windows event logs come in via a modular input these days. The modular input sends in the pre-parsed time as it comes from the Windows event log APIs. So Splunk does not have to do any timestamp parsing, and therefore we don't get the date_* fields.

Cross-links to other similar questions:

http://answers.splunk.com/answers/30822/date-hour-not-present-in-wineventlogs.html
http://answers.splunk.com/answers/92087/default-fields-are-not-visible.html#comment-92199

wkupersa
Path Finder

The useful side-effect is useful....And missed when not there. I don't know why my initial search didn't reveal those other questions. Thank you very much for your answer!

jfunderburg
Explorer

if this is true, why do my splunk servers running windows 2012r2 create the date_* field for there own eventlogs? they are using the same props.conf and Splunk_TA_windows app. when I seach there windows log, they return date_* fields. None of my universal forwarders on windows servers 2012r2 or otherwise or my windows 7 clients do. the only difference I can find is all my servers (Search heads, indexers, mast indexer, deployment server) are running splunk enterprise. My other systems are running universal forwarders. I have used universal forwarder 6.4.0, 6.5.0 and am now trying 7.0.0. it would make sense if NONE of my windows events gave date_* fields.... but they do. I really would prefer this work to take load of search head parsing days and hours from search to return non-business hour logins. I can do this using eval to create the fields but it is EXTREMELY slow and search head intensive as it has to return all results the evaluate and parse them. Vice only returning the valid events from the Index using date_wday and date_hour.

somesoni2
Revered Legend

Do you see any difference in the timestamp in Splunk (_time) and in raw data?

0 Karma

wkupersa
Path Finder

Just in the format: _time = 2015-03-10 09:09:59 and _raw = 03/10/2015 09:09:59 AM....

0 Karma

wkupersa
Path Finder

No love here. Can others at least confirm that they experience this issue?

0 Karma
Get Updates on the Splunk Community!

Introducing Ingest Actions: Filter, Mask, Route, Repeat

WATCH NOW Ingest Actions (IA) is the best new way to easily filter, mask and route your data in Splunk® ...

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...