topic Re: AM PM being ignored by TIME_FORMAT in props.conf in Getting Data In

AM PM being ignored by TIME_FORMAT in props.conf

phoenixdigital — Tue, 25 Sep 2012 05:06:56 GMT

Hi All,

Having an issue importing the following data.

UID, In Date, Update Time, Vol, Corr Vol
453,May 1 2012 6:00AM,May 2 2012 3:24PM,133,223.000000000
453,May 1 2012 7:00AM,May 2 2012 3:24PM,104,175.000000000
453,May 1 2012 8:00AM,May 2 2012 3:24PM,90,152.000000000

props.conf

[Flow]
KV_MODE = none
SHOULD_LINEMERGE = false
TIME_FORMAT = %b %d %Y %I:%M%p
TRANSFORMS-filterprices = filterFlowData, filterFlowHeaderRow
REPORT-extracts = FlowDataCsvExtract

transforms.conf

[filterFlowHeaderRow]
REGEX = ^UID(.*)
DEST_KEY = queue
FORMAT = nullQueue

[filterFlowData]
REGEX = (.*)
DEST_KEY = queue
FORMAT = indexQueue

[FlowDataCsvExtract]
DELIMS = ","
FIELDS = "UID", "In Date", "Update Time", "Vol", "Corr Vol"

It appears Splunk is retrieving the first time but completely ignoring the AM PM even though I am specifying it with the %p argument.

Anyone have any clues how to persuade Splunk to not stop parsing the date too soon?

Re: AM PM being ignored by TIME_FORMAT in props.conf

kristian_kolb — Tue, 25 Sep 2012 08:13:55 GMT

Hi,

Two things that MAY improve the situation:

a) Set the TIME_PREFIX = ^\d+,

b) Set the MAX_TIMESTAMP_LOOKAHEAD=18

I have seen (on strftime.net) that %l (lowercase L) should be the 12-hour format without leading zeroes. Other sources claim that %I allows optional leading zeroes. Unfortunately I haven't found a definite list of the supported strftime/strptime formatting that Splunk accepts/recognizes.

Also, on a side note, I don't think you need the filter for the parsing queue, since that is the default. You only need to filter out the header row to the null queue.

Hope this helps,

Kristian

Re: AM PM being ignored by TIME_FORMAT in props.conf

phoenixdigital — Tue, 25 Sep 2012 20:30:55 GMT

Thanks for the tip it seems the TIME_PREFIX did the trick and its picking up the AM PM now.

I tested both %I and %l (lowercase L) and both behaved in the same manner.

Good to know about the filter as well. Nothing better than simplifying things.

Thanks again