Splunk does not pickup custom timestamp prior to year 2012

[raja21]

Splunk Version: 7.1

I have a custom time stamp field in my JSON records in this format, "_timestamp"="1/3/2013 10:12:56.000 PM".
On uploading the record Splunk picks up the custom timestamp and assigns it to _time, however when the year is before 2012 i.e.
"_timestamp"="1/3/2012 10:12:56.000 PM" or "_timestamp"="1/3/2011 10:12:56.000 PM" splunk throws an error that it cannot use regex to parse the timestamp.
I have been beating my head around this and would love to know the solution for this.

P.S. i have not changed any configuration in props.conf, splunk automatically picks up the custom timestamp as it is in the exact format.

[FrankVl · Accepted solution]

I guess that behavior is because by default MAX_DAYS_AGO is set to 2000, which is ~5,5 years, which indeed at this moment results in Splunk accepting timestamps from 2013, but not from 2012.

So if you want to indeed ingest that old data, you will have to configure Splunk to allow that.

[raja21]

Hi @FrankVI thanks for your suggestion.
Can you please tell me how and where should i set MAX_DAYS_AGO?

[niketn]

It should be setup in props.conf for the sourcetype you are trying to add.

If you are testing with sample data (Upload a test file), you can test this out while setting the sourcetype under the Advanced Settings section by manually typing the MAX_DAYS_AGO and the number of days to identify the oldest event.

[raja21]

@niketnilay I got it working, thanks 🙂

[niketn]

@raja21, glad you found it working. Do up vote the comments that helped 🙂

[niketn]

@raja21 look into MAX_DAYS_AGO as suggested by FrankVI. Even while uploading couple of dummy events you can check the behavior whether correct timestamp is identified by changing MAX_DAYS_AGO or not.

[raja21]

Can you please tell me how and where should i set MAX_DAYS_AGO?

[xpac]

Best practice hint: Always set up timestamp recognition for your data, don't let Splunk guess. It will not only save you trouble like this, but maybe even more in the future - because Splunk might recognize that timestamp as "day 1 of month 3", as well as "day 3 of month 1".

[raja21]

I'm practicing the standard format of %m/%d/%y H:M:S.000 AM/PM so it should always pick it up in the correct format i guess.