Splunk can't handle old timestamps

[msarro]

Greetings everyone. I am receiving a gamut of old files, some of which contain test data showing records from 1970. Splunk is indexing them using the file's modtime and not the timestamp, which is quickly diluting real data. Why doesn't splunk index them using the correct timestamp as shown in the file?

[msarro]

Please don't be offended, but the fix mentioned below appears to be what is needed. The CDR is more than 300 fields long, and would take a LONG time to manually sanitize. The benefit would be limited since the below excerpt indicates that splunk only accepts timestamps in the past 2000 days - and the data we're seeing is a limited portion of the total sample.

[araitz]

Just one sanitized log sample would suffice. It is very hard to conjecture our way to solving your problem.

[msarro]

I would love to, however this is CDR data, and under federal regulations I can't share it samples without sanitizing them. However I am using 4.3. I am using a configured sourcetype, as well as 4 transforms. Although I believe the next poster's solution is the one I need.

[Ayn · Accepted solution]

You'll likely want to increase MAX_DAYS_AGO in props.conf.

MAX_DAYS_AGO = <integer>
* Specifies the maximum number of days past, from the current date, that an extracted date  can be valid.
* For example, if MAX_DAYS_AGO = 10, Splunk ignores dates that are older than 10 days ago.
* Defaults to 2000 (days), maximum 10951.
* IMPORTANT: If your data is older than 2000 days, increase this setting.

[bwooden]

It would be helpful to those wanting to help if you would provide the version of Splunk you're using, a few sample lines of source data, and whether you are using auto sourcetyping or have configured your own sourcetype.