Getting Data In

Why Splunk interpret a false timestamp?

marcokrueger
Path Finder

I configured my data import with a timeformat of %6N and all works fine. Sometimes the event comes with a %3N timestamp and I expect Splunk ignore it, because interpreted as a %6N Timestamp it is to far in the past to be conform with the MAXDAYSAGO configuration.
Why Splunk accept it contrary to the configuration?

best regards Marco

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

If you specify a TIMEFORMAT for the timestamp, and one event does not match exactly the format, then splunk will default to the automatic detection of the timestamp for this particular event.

For the MAX_DAYS_AGO, please provide a sample of your events and the props.conf to see what may be happening ?

yannK
Splunk Employee
Splunk Employee

I do not know it this is related, but :
The TIME_FORMAT=%6N does not match timestamp 1382140800002914

The format you have is the concatenation from the epoch time in seconds, followed by milliseconds on 6 digits without a separator.

try with TIME_FORMAT=%s%6N

0 Karma

marcokrueger
Path Finder

Now I have configured MAX_DAYS_AGO to test the behaviour:
here my examples:
{"requestTime":1382140800002914,... # correct interpreted as %6N
{"requestTime":1382140800,... # correct interpreted as default, because date is before TODAY-MAX_DAYS_AGO
{"requestTime":1382140800019, # not correct interpreted, interpreted as %3N (10/19/13 2:00:00.019 AM), but in %6N I expect the 01/17/1970 00:55:40.800 AM (this is ~16022 days ago)

BREAK_ONLY_BEFORE=^ {
MAX_TIMESTAMP_LOOKAHEAD=16
SHOULD_LINEMERGE=false
TIME_FORMAT=%6N
TIME_PREFIX={"requestTime":
TRUNCATE=0
MAX_DAYS_AGO=170000

0 Karma

marcokrueger
Path Finder

I think, perhaps it is a fallback behaviour, but I cant find it in the docs. In this case splunk does exact whjat I want, but I want to be sure, that it is correct and demerministic.

0 Karma
Get Updates on the Splunk Community!

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...

Stay Connected: Your Guide to October Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...