Issues with timestamps

[MickSheppard]

I'm trying to get Splunk to index the output from the Connect:Enterprise cmulist command. I run the command periodically and filter out the rubbish just leaving me with the lines that count. This gives me one line per entry and with the following date format from around character 65.

11/07/06-07:00

This is yy/mm/dd-HH:MM so I've set TIME_FORMAT for this data source to %y/%m/%d-%H:%M. This appeared to be working but I noticed some missing records and finally found them away in 2006.

I set the TIME_FORMAT value in my etc/system/local/props.conf.

I'm also seeing that occasionally it doesn't bother using the date/time in the record instead it uses current time. It then clumps all of the lines together in a single entry.

Anyone any tips on how I fix these problems, short of using perl to reformat the input into a format that Splunk can reliably process?

[dwaddle · Accepted solution]

Have you experimented with MAX_TIMESTAMP_LOOKAHEAD and TIME_PREFIX as well? These two help to key Splunk in on a specific area of the event text for which to apply your TIME_FORMAT.

If you can provide some sample (redacted) data, that could help narrow down why Splunk isn't picking up the time like you were expecting.

You could always pop on #splunk on EFNet IRC and discuss this there. Again, with some sample data, this should be fairly easy to work out.

[MickSheppard]

I played with MAX_DAYS_AGO and MAX_DAYS_HENCE and now it seems to be working without any issues.