I configured my data import with a timeformat of %6N and all works fine. Sometimes the event comes with a %3N timestamp and I expect Splunk ignore it, because interpreted as a %6N Timestamp it is to far in the past to be conform with the MAXDAYSAGO configuration.
Why Splunk accept it contrary to the configuration?
best regards Marco
If you specify a TIMEFORMAT for the timestamp, and one event does not match exactly the format, then splunk will default to the automatic detection of the timestamp for this particular event.
For the MAX_DAYS_AGO, please provide a sample of your events and the props.conf to see what may be happening ?
I do not know it this is related, but :
The TIME_FORMAT=%6N
does not match timestamp 1382140800002914
The format you have is the concatenation from the epoch time in seconds, followed by milliseconds on 6 digits without a separator.
try with TIME_FORMAT=%s%6N
Now I have configured MAX_DAYS_AGO to test the behaviour:
here my examples:
{"requestTime":1382140800002914,... # correct interpreted as %6N
{"requestTime":1382140800,... # correct interpreted as default, because date is before TODAY-MAX_DAYS_AGO
{"requestTime":1382140800019, # not correct interpreted, interpreted as %3N (10/19/13 2:00:00.019 AM), but in %6N I expect the 01/17/1970 00:55:40.800 AM (this is ~16022 days ago)
BREAK_ONLY_BEFORE=^ {
MAX_TIMESTAMP_LOOKAHEAD=16
SHOULD_LINEMERGE=false
TIME_FORMAT=%6N
TIME_PREFIX={"requestTime":
TRUNCATE=0
MAX_DAYS_AGO=170000
I think, perhaps it is a fallback behaviour, but I cant find it in the docs. In this case splunk does exact whjat I want, but I want to be sure, that it is correct and demerministic.