topic TIME_FORMAT, it's May, not January in Getting Data In

TIME_FORMAT, it's May, not January

howyagoin — Mon, 02 May 2011 00:02:10 GMT

Overnight I noticed that my Splunk was suspiciously empty for a specific CSV file which was read in daily.

Upon further investigation it turned out this was due to the fact that the file now has data for May 1, or, 01/05/2011....the CSV auto-parsing saw the format for the last half of last month and worked out, properly, that it was Day/Month/Year, (16/04/2011 was pretty obvious) but now, well, it's a bit fuzzier.

Reading up in the manuals, it looks like the answer is to set the TIME_FORMAT in props.conf, which I have done with:

[source::/A/B/C/data/MyFile*]
TIME_FORMAT = %d/%m/%y %H:%M

The source is already being indexed with a inputs.conf entry for "/A/B/C/data"

The data looks like:

3812305781230123    , 7773213          , 9099, B, 75, INTERNET, 01/05/2011 23:58

However, Splunk is still seeing this as data for January.

What should I do to fix this?

Thanks!

Re: TIME_FORMAT, it's May, not January

dmlee — Mon, 02 May 2011 02:12:26 GMT

Hi, I think you should use %Y instead of %y , because the year is "2011" (four digits).
and , if possible, you can use TIME_PREFIX to tell splunk where is the exact date field you want splunk to recognize.

Re: TIME_FORMAT, it's May, not January

howyagoin — Mon, 02 May 2011 03:40:44 GMT

Hah, thanks, great catch on the %Y. That didn't actually fix my issue, but, the TIME_PREFIX seems to get me closer. There are six fields, comma separated, before my date/time field, so I just need to work out the regex on this -- as some of the fields are empty sometimes, othertimes not (annoying).

Re: TIME_FORMAT, it's May, not January

howyagoin — Mon, 28 Sep 2020 09:31:02 GMT

For the benefit of those reading who may want a concrete example:



TIME_PREFIX = ^(?:[^\,]+,){6}\s*

TIME_FORMAT = %d/%m/%Y %H:%M