TIME_FORMAT, it's May, not January

[howyagoin]

Overnight I noticed that my Splunk was suspiciously empty for a specific CSV file which was read in daily.

Upon further investigation it turned out this was due to the fact that the file now has data for May 1, or, 01/05/2011....the CSV auto-parsing saw the format for the last half of last month and worked out, properly, that it was Day/Month/Year, (16/04/2011 was pretty obvious) but now, well, it's a bit fuzzier.

Reading up in the manuals, it looks like the answer is to set the TIME_FORMAT in props.conf, which I have done with:

[source::/A/B/C/data/MyFile*]
TIME_FORMAT = %d/%m/%y %H:%M

The source is already being indexed with a inputs.conf entry for "/A/B/C/data"

The data looks like:

3812305781230123    , 7773213          , 9099, B, 75, INTERNET, 01/05/2011 23:58

However, Splunk is still seeing this as data for January.

What should I do to fix this?

Thanks!

[dmlee · Accepted solution]

Hi, I think you should use %Y instead of %y , because the year is "2011" (four digits).
and , if possible, you can use TIME_PREFIX to tell splunk where is the exact date field you want splunk to recognize.

[howyagoin]

For the benefit of those reading who may want a concrete example:

TIME_PREFIX = ^(?:[^\,]+,){6}\s*

TIME_FORMAT = %d/%m/%Y %H:%M

[howyagoin]

Hah, thanks, great catch on the %Y. That didn't actually fix my issue, but, the TIME_PREFIX seems to get me closer. There are six fields, comma separated, before my date/time field, so I just need to work out the regex on this -- as some of the fields are empty sometimes, othertimes not (annoying).