Getting Data In

Why Splunk interpret a false timestamp?

marcokrueger
Path Finder

I configured my data import with a timeformat of %6N and all works fine. Sometimes the event comes with a %3N timestamp and I expect Splunk ignore it, because interpreted as a %6N Timestamp it is to far in the past to be conform with the MAXDAYSAGO configuration.
Why Splunk accept it contrary to the configuration?

best regards Marco

Tags (2)
0 Karma

yannK
Splunk Employee
Splunk Employee

If you specify a TIMEFORMAT for the timestamp, and one event does not match exactly the format, then splunk will default to the automatic detection of the timestamp for this particular event.

For the MAX_DAYS_AGO, please provide a sample of your events and the props.conf to see what may be happening ?

yannK
Splunk Employee
Splunk Employee

I do not know it this is related, but :
The TIME_FORMAT=%6N does not match timestamp 1382140800002914

The format you have is the concatenation from the epoch time in seconds, followed by milliseconds on 6 digits without a separator.

try with TIME_FORMAT=%s%6N

0 Karma

marcokrueger
Path Finder

Now I have configured MAX_DAYS_AGO to test the behaviour:
here my examples:
{"requestTime":1382140800002914,... # correct interpreted as %6N
{"requestTime":1382140800,... # correct interpreted as default, because date is before TODAY-MAX_DAYS_AGO
{"requestTime":1382140800019, # not correct interpreted, interpreted as %3N (10/19/13 2:00:00.019 AM), but in %6N I expect the 01/17/1970 00:55:40.800 AM (this is ~16022 days ago)

BREAK_ONLY_BEFORE=^ {
MAX_TIMESTAMP_LOOKAHEAD=16
SHOULD_LINEMERGE=false
TIME_FORMAT=%6N
TIME_PREFIX={"requestTime":
TRUNCATE=0
MAX_DAYS_AGO=170000

0 Karma

marcokrueger
Path Finder

I think, perhaps it is a fallback behaviour, but I cant find it in the docs. In this case splunk does exact whjat I want, but I want to be sure, that it is correct and demerministic.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...