Splunk Search

Why do some events display date fields and other don't?



I noticed that one of my custom feeds has date fields (date_hour, date_mday...), but other ones, which are native to Splunk (perfmon) do not provide these fields. What is the reason for this?

Tags (3)

Esteemed Legend

In most Splunk datasets the ‘date_ ’ fields almost always exist and because of this you may not notice that sometimes they don’t! Most people naturally assume these fields are default fields (like ‘time’) that are always there. The ‘date’ fields are a (very useful) side-effect from Splunk’s timestamping process. If Splunk does not (have to) parse an event to set its timestamp then the ‘date_’ fields will not exist. Windows event logs, for example, now come in via a modular input that is designed to use the pre-parsed time as it comes from the Windows event log APIs, obviating the need for Splunk to do any timestamp parsing, therefore the ‘date_’ fields are not created, do not exist, and are unavailable for our use.

You can force them to exist when they do not with something like this:

index=* | eval date_wday=lower(coalesce(date_wday,strftime(_time,"%A"))) | stats count by date_wday



do you want to say that if data come via HEC input Splunk does not show date_* fields in search-time? I though date_* fields are somehow automatically populate when reading _time field during search-time.

All my events that came via HEC (from Outcold collector) have cooked timestamp correctly. So Splunk is not forced to do any parsing on HF where HEC endpoint catches events. So timestmaps are fine. But during search-time I do not see any date_* fields and I think we all need them. They are so practical.

Do you want to say Splunk indexes date_* fields during index-time?


Esteemed Legend

The date_* fields are only generated when necessary to do Time zone normalization at index time so they only exist about 2/3 of the time and they should almost never be used because they are UTC-based (remember, these are index-time, not search-time) and do not adjust to the user's Time zone setting.

0 Karma


If the date and time appear in the raw input data file, then the date_* fields usually exist. You should be able to look at the events that are returned from a search and see if the data contains date or time information.

However, many scripted inputs do not actually have the date and time in the raw input - the timestamp is based on when the script was run or when the perfmon data was collected. This is also often true for network inputs as well. In this case, you will not see any date_* fields.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...