Getting Data In

Why Splunk interpret a false timestamp?

marcokrueger
Path Finder

I configured my data import with a timeformat of %6N and all works fine. Sometimes the event comes with a %3N timestamp and I expect Splunk ignore it, because interpreted as a %6N Timestamp it is to far in the past to be conform with the MAXDAYSAGO configuration.
Why Splunk accept it contrary to the configuration?

best regards Marco

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

If you specify a TIMEFORMAT for the timestamp, and one event does not match exactly the format, then splunk will default to the automatic detection of the timestamp for this particular event.

For the MAX_DAYS_AGO, please provide a sample of your events and the props.conf to see what may be happening ?

yannK
Splunk Employee
Splunk Employee

I do not know it this is related, but :
The TIME_FORMAT=%6N does not match timestamp 1382140800002914

The format you have is the concatenation from the epoch time in seconds, followed by milliseconds on 6 digits without a separator.

try with TIME_FORMAT=%s%6N

0 Karma

marcokrueger
Path Finder

Now I have configured MAX_DAYS_AGO to test the behaviour:
here my examples:
{"requestTime":1382140800002914,... # correct interpreted as %6N
{"requestTime":1382140800,... # correct interpreted as default, because date is before TODAY-MAX_DAYS_AGO
{"requestTime":1382140800019, # not correct interpreted, interpreted as %3N (10/19/13 2:00:00.019 AM), but in %6N I expect the 01/17/1970 00:55:40.800 AM (this is ~16022 days ago)

BREAK_ONLY_BEFORE=^ {
MAX_TIMESTAMP_LOOKAHEAD=16
SHOULD_LINEMERGE=false
TIME_FORMAT=%6N
TIME_PREFIX={"requestTime":
TRUNCATE=0
MAX_DAYS_AGO=170000

0 Karma

marcokrueger
Path Finder

I think, perhaps it is a fallback behaviour, but I cant find it in the docs. In this case splunk does exact whjat I want, but I want to be sure, that it is correct and demerministic.

0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...