BREAK_ONLY_BEFORE_DATE=true is not working

[mukuru74]

Here is my log sent from an UF to and Indexer:

2019-09-16 09:37:00 Fetching ISS data
'issfiles/sampleFile.tmp' -> 'issfiles/sampleFile.new'
2019-09-16 09:37:04 Fetch of ISS data completed successfully
2019-09-16 09:37:04 Processing ISS data
Directory: processISS/issfiles
Sucessfully parsed status log file
2019-09-16 09:37:04 Processed ISS data

Here is my props.conf on the indexer:

MAX_TIMESTAMP_LOOKAHEAD=30
BREAK_ONLY_BEFORE_DATE=true
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
TZ=UTC

I don't understand why the event 'issfiles/sampleFile.tmp' -> 'issfiles/sampleFile.new' is not merged with the previous event 2019-09-16 09:37:00 Fetching ISS data

Here is what I see from search:

Can anybody help?

[mukuru74]

Hi Yorokobi
Between the first line and the arrant line are a carriage return and line feed.

[mukuru74]

Hi Yorokobi
Thak you for your feedback.
I have tried your suggestion, but it's still not working.
Here is my new props.conf file:
[ptss-dashboardLog]
TIME_FORMAT = %F %T
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}
TZ=UTC
As you can see on the screenshot below, it's still creating a new event for the string "'issfiles/sampleFile.tmp' -> 'issfiles/sampleFile.new'" even though there is no timestamp preceding this event.

[Yorokobi]

Where does that errant line live in the raw data? At the top of the file (line 1)?

[mukuru74]

Hi have changed the props.con as follow but it's still not working.
TIME_FORMAT = %F %T
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}
TZ=UTC

As you can see, Splunk has created an event for the yellow marked string eventhough there is no date preceding it.

[Yorokobi]

BREAK_ONLY_BEFORE_whatever options should be avoided when a proper LINE_BREAKER entry can be used.

[some_sourcetype]
TIME_FORMAT = %F %T
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}

[mukuru74]

Thank you for your reply.
But it’s still not working.

[richgalloway]

The TIME_FORMAT setting does not match your sample data. Try
TIME_FORMAT = %Y-%m-%d %H:%M:%S