Why has my Indexer stopped indexing after indexing data with future time stamp from Splunk DB Connect?

[karthikklv]

Hi All,
Splunk Indexer stopped indexing data after indexing data with future time stamp in from Splunk DB Connect.

Here is my setup.
I have Table A who's The 'UPDATE_DT' column is mapped as the 'Rising column' and 'Timestamp' attributes. A conversion issue populated data with a future UPDATE_DT into the table and the Splunk Indexer stopped indexing post indexing data with future UPDATE_DT.

I expected the indexer to start consuming data after I changed the 'checkpoint value' to current date minus 2 (the day Splunk indexer stopped) and 'Timestamp' to 'current index time'. But, no luck.

'UPDATE_DT' column is the only column which can be used as 'Rising column' and data conversion issues are bound to happen.

How can I
1) Repair the meta data so that Splunk Indexer start consuming data
2) Fix the setup so that it doesn't stop indexing data when events with future data arrive.

Appreciate your help. Thx.

[woodcock]

The right thing to do is fix your timestamp. It may be as simple as applying the appropriate timezone details but it makes no sense to deliberately send data to Splunk when you know that the time is invalid. You have GOT to fix _time and the best place to fix it is in the system from which you are getting it (not in Splunk). Since this is coming from a DB, fix it there (or translate it into the correct format in the SQL, not in Splunk).

As far as forcing splunk to eat future data, check out my answer here:

https://answers.splunk.com/answers/547506/handling-future-dates-from-csv.html#answer-547508