Splunk Search

Why is the time off by a hundredth of a second?

ddrillic
Ultra Champion

We have a case in with the time is off by a hundredth of a second for many events of a certain sourcetype -

alt text

What can it be? We don't see parsing errors in the logs...

Tags (1)
0 Karma
1 Solution

skoelpin
SplunkTrust
SplunkTrust

It's because you're missing your base configs in your props.conf file. Whenever you create a new sourcetype, you should ALWAYS write base configs for that sourcetype so Splunk knows how to linebreak and timestamp your events. This is why I always urge customers to use the least amount of sourcetypes as possible and fit them to the format of the data.

These base configs include these 6 attributes below. The first one identifies where the timestamp is in the event (i.e. the beginning of the event), the second one tells Splunk how the time format is setup so Splunk know's how to read it (difference between 1/01/2018 and 2018-Jan-01) and the third one tells Splunk how far to look into the event to find your timestamp. So most likely, you are not explicitly telling Splunk how to read your time format which is why its not correct. If you were to add the base configs to your indexers via the props.conf I listed below then it will read them correctly. *You will need to write the line_breaker one yourself and add the kv_mode=json. Just a heads up, but your hour is incorrect too, it should show 8:38pm and not 2:38pm

[<SOURCETYPE>]
TIME_PREFIX=message:\s
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3Z
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER=([\n\r]+)(\w+\/\w+\/\d+)
SHOULD_LINEMERGE=0
TRUNCATE=99999

View solution in original post

skoelpin
SplunkTrust
SplunkTrust

It's because you're missing your base configs in your props.conf file. Whenever you create a new sourcetype, you should ALWAYS write base configs for that sourcetype so Splunk knows how to linebreak and timestamp your events. This is why I always urge customers to use the least amount of sourcetypes as possible and fit them to the format of the data.

These base configs include these 6 attributes below. The first one identifies where the timestamp is in the event (i.e. the beginning of the event), the second one tells Splunk how the time format is setup so Splunk know's how to read it (difference between 1/01/2018 and 2018-Jan-01) and the third one tells Splunk how far to look into the event to find your timestamp. So most likely, you are not explicitly telling Splunk how to read your time format which is why its not correct. If you were to add the base configs to your indexers via the props.conf I listed below then it will read them correctly. *You will need to write the line_breaker one yourself and add the kv_mode=json. Just a heads up, but your hour is incorrect too, it should show 8:38pm and not 2:38pm

[<SOURCETYPE>]
TIME_PREFIX=message:\s
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3Z
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER=([\n\r]+)(\w+\/\w+\/\d+)
SHOULD_LINEMERGE=0
TRUNCATE=99999

skoelpin
SplunkTrust
SplunkTrust

@ddrillic did this help?

0 Karma

ddrillic
Ultra Champion

Thank you @skoelpin !!

adonio
Ultra Champion

please verify,
in this particular data, you should have in props.conf the
verify that the field set for timestamp is: "message" (in this data set)
also pay attention, 2:00pm is 14:00 not 20:00
I assume that fixing your props.conf will solve it

ddrillic
Ultra Champion

Thank you @adonio !

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...