Index Time VS Actual Occurs Time

[jackyc]

Hi all,

I have a month (2010-Nov) SAR reports (30 copies) for my host which I want to import them to the Splunk server. For testing purpose, I first import one SAR report to the Splunk and it can be successfully imported. But the event time is today not two months ago, can I change the index time back to the actual occurs time? Since I need to search for (2010-Nov)'s SAR report. I found maillog didn't have this issue..

Many thx!

BR, Jacky.

[cyndiback]

Hi Jacky,
I encountered the same issue today when indexing old data into Splunk but wanting to preserve the actual time as index time.

Copy of the logs I'm indexing:

• ....change_time: 2011-11-04 10:30:27, view_rfc_status, 1803, 17, Approve, 137, John Doe, 2243
• ....change_time: 2011-11-04 10:30:47, view_rfc_status, 1803, 17, Approve, 137, John Norris, 2243
• ....change_time: 2011-11-04 10:40:13, view_rfc_status, 1806, 17, Approve, 142, Chuck Norris, 2246
• ....change_time: 2011-11-04 12:17:39, view_rfc_status, 1807, 16, Pending Approval, 148, Chuck Norris, 2247

The correct timestamp should be the after change_time: 2011-01-04 10:30:27 but if I indexed these today Splunk would mark them as 2012-01-07 12:10:00 PM

To always use the time in the log I made the following changes:

• On the Splunk indexer edit the local props.conf (if linux server file is in /opt/splunk/etc/system/local/props.conf)
• Create a stanza for the specific source
• Tell Splunk what comes before the timestamp you want to use - In my case the timestamp is after "change_time: "
• Tell Splunk what format the datetime is in

Copy of stanza in Props.conf

[source::/opt/splunk/bin/scripts/rfc_status.sh]  #specific source
     TIMEPREFIX="changetime:  "     #look for time after this text
     TIME_FORMAT=%Y-%m-%d %H:%M:%S  #this is how time is formatted

Followed Splunk Docs: http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition

I wanted to clean up the logs I had already indexed incorrectly so my whole process was to (NOTE depending on your setup this process may not work for you):

• Disable indexing for the specific source while making the props.conf changes
• Delete the old data for the specific source *****Careful you do not delete ALL logs from host.
  
• Save the props.conf changes

• Reload config changes in props.conf by typing the following search string in Splunk Web:

  | extract reload=T

• Enabled indexing for the specific source

This is what I did I don't know if there are easier ways to do this.

[Drainy]

One note, as this is an index time change you will need to restart Splunk to reload the relevant changes in props.conf. The extract reload=T command will only reload search time extractions.