date_hour not present in WinEventLogs

[kristian_kolb]

Trying to write a search that list events happening outside office hours, across a bunch of sourcetypes - however, the 'date_hour' is not available/reportable/searchable for Windows Event Logs. It just isn't there among the available fields.

e.g.

WORKS FINE: sourcetype=linuxsecure root (date_hour < 8 OR date_hour > 17)

WORKS NOT: sourcetype=wineventlog:security administrator (date_hour < 8 OR date_hour > 17)

The following search confirmed that date_hour is not extracted for WinEventLogs, i.e. the date_hour column is empty on the WinEventLog row.

* | dedup sourcetype | table date_hour, sourcetype

Is there something that could be done about this? It seems like the problem is within the way Splunk treats the WinEvt-logs, rather than something to be done at search time.

The Docs only state that default fields, such as date_*, will be extracted only if the information is present within the event itself. When looking at Windows events, they sure do have timestamps in them. Any help or clarification appreciated.

Running 4.2.1 on RH, forwarders are 4.2.2 on multiple OS's. Other events (various application logs) collected by Win-forwarders contain the date_* default fields....

Thanks in advance,

Kristian

[kristian_kolb · Accepted solution]

The word from the support team is that this is a known limitation of how Splunk treat the WinEventLogs when grabbed through a forwarder (even though the docs do not mention this).

The suggested workaround is the same as Ayn mentioned;

*| eval date_hour = strftime(_time, "%H")

although I suspect one could make a permanent field extraction if date_hour for WinEventLogs is needed in more than just a few places.

Kristian

[jaxjohnny2000]

The date_hour is a Splunk default field which have timestamp information in that as generated by their respective systems. These fields come over unmodified. the _time field is presented to the user the preferences for their local time zone for display purposes

https://docs.splunk.com/Documentation/Splunk/7.2.6/Knowledge/Usedefaultfields

For example,

Assuming the Windows Server is in Pacific Standard time, and the user(me) is in Eastern time.

if the event occurred on the server at 07:37AM PST, then my time will show 10:37AM EST

So the code: *| eval date_hour = strftime(_time, "%H") will production the following output

_time = 10:37 AM
date_hour = 10
actual event time (which I want to display) = 07:37AM

So the code above merely shows the time value using the offset by the user. I want to see 07 in the date_hour field, like I would if syslog sent a record. The code does indeed create a date_hour field, but not the same date_hour which Splunk generates as a default field.

the date_* fields are only missing for Windows Events. They are visible for F5, Netscaler, and Cisco.

We have servers in all four time zones, and Hawaii.

Splunk Support tells me this unfortunately:

Unfortunately, the default date-time fields in Splunk doesn't work for Windows Event Logs as it uses API to ingest data rather the regular data ingestion pipeline.

The link below explains default fields:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Knowledge/Usedefaultfields

[Ayn]

c,

I'd say this behaviour is a bug, and you should file a case with support.

That said, there are ways to work around it in the meantime by extracting the date_hour (and other) fields you need from _time instead by using eval and strftime.

sourcetype=wineventlog:security administrator | eval date_hour=strftime(_time, "%H") | where (date_hour<8 OR date_hour>17)

I took the liberty to rewrite the condition using OR instead of AND, as date_hour<8 AND date_hour>17 will never match.

Cheers,

P

[kristian_kolb]

I have not had any real experience with using WMI to fecth WinEventLogs, but the little I have seen indicates that the date_hour field is indeed extracted properly.

[ehoward]

I hope this is fixed soon. WMI data sources are certainly critical enough that we should not have to resort to hacks like this.

[kristian_kolb]

P!

Thanks for pointing that one out. That's what happens when you copy someone elses code example code 🙂 The character validation was messing with my dangerous HTML characters, so I just copied somebody elses code example for searches WITHIN office hours - hence the reversed position of < and > (first edit) and the 'AND' (second edit)

The problem still persists, and I will file a support case.

c.

[Ayn]

I just tested this on a Universal Forwarder v4.2.3 running on Windows 7 (Swedish language for what it's worth) forwarding to a v4.2.3 indexer on Ubuntu and can confirm that I'm not getting any date_* fields either. Normally this will happen if Splunk can't obtain a valid timestamp from an event, but I'm getting correct timestamps even for old events that were generated before I turned the log monitor on, so Splunk obviously finds and uses the timestamp.

[kristian_kolb]

Have not updated the RH splunk indexer yet, but I had a look at a couple of other Windows indexers, and date_hour was not available/extracted on either of them when fetching the logs through forwarders.

One of the indexers was also fetching EventLogs through WMI, and on those the date_* fields were extracted. Anyone who can shed light on this?

Thanks,

Kristian

[gekoner]

Kristian, this is not an answer but I can confirm that you should have a date_hour Field for your WinEventLogs.
I'm guessing you have something interfering with your default field extractions.
Can you upgrade to 4.2.3 on your Indexer?