I'm having some issue with the configuration on one of my source. Even after configuring the props.conf, events are not broken properly.
Format of my source :
09:39:37.889 INFO [main] Instantiated BDPeriodicAgent - o.i.p.m.b.s.impl.BDPeriodicAgent:57 09:39:37.921 DEBUG [main] Started meeting lifecycle agent to run every 36000 s - o.i.p.w.m.bd.servlet.BDInitServlet:64
My props.conf is the following:
MAX_TIMESTAMP_LOOKAHEAD = 150 NO_BINARY_CHECK = 1 TIME_FORMAT = %H:%M:%S.%3N
When I try to add the file from the Splunk interface and add this configuration to the "Advanced mode", events are shown properly. But when the same file is coming from the forwarder it looks like the props.conf is not taken in account and event are not split on the timestamp
There are a couple of possible things going on here. If the forwarder in question is what's known as a heavy forwarder (that is, a full instance of splunk with an outputs.conf) it may be parsing the events (and handling event breaking) before it ever gets to the indexer.
Assuming that's not the case, I've heard it said that Splunk wants to capture both a date and a time with TIME_PREFIX, and if it can't, then it assumes it got the wrong answer and doesn't consider what it found to be a valid "_time", which is typically how the event boundary is determined.
I'd go with @somesoni2's answer above, as the quick way to fix the problem.