I am trying to apply a custom TIME_FORMAT to a wildcarded source in props.conf, but Splunk doesn't seem to be applying it (i.e. Splunk still shows the wrong _time in my search against this source).
What am I doing something wrong?
Example source:
albatross-b827e88ab79c.vsat.20150209.speedtest.csv
My entry in props.conf:
[source::*speedtest.csv]
TIME_FORMAT = %Y%m%d%H%M
TIME_PREFIX = ^
Sample event:
201502062345,albatross-b827ebe142fe.vsat,12.23,Mb,3.30,Mb
I would add a MAX_TIMESTAMP_LOOKAHEAD = 20 ... (it has been helpfull to me in the past) just to make sure SPlunk does not reads to long into the file, since it is in the beginning.
I can also recommend the "Data preview" function that exists in SPlunk now ... it will display / help you with this sorts of problems.
I believe that there is a bug with using a wildcard with a source in props.conf -- the wildcard pattern only works in some scenarios. We have seen issues with * specifically.
Support acknowledges but has not fixed. Documentation should be updated to reflect not to use asterisk as a wildcard, use ... instead.
I would add a MAX_TIMESTAMP_LOOKAHEAD = 20 ... (it has been helpfull to me in the past) just to make sure SPlunk does not reads to long into the file, since it is in the beginning.
I can also recommend the "Data preview" function that exists in SPlunk now ... it will display / help you with this sorts of problems.
Since your timestamps are the beginning of each event, you don't need TIME_PREFIX.
Right, it still wouldn't recognize it without TIME_PREFIX though. I am starting to think you can't apply TIME_FORMAT to a source because I couldn't find an example in the props.conf.spec.
I worked around the issue by re-indexing the data under a new sourcetype I created with the time_format settings.