Ingesting custom log with custom sourcetype but trouble getting event breaks properly configured?

[g9s0x1]

I have a script that creates a custom log file to gather all Splunk certs and uses openssl to print out all of the details about each cert (.pem files). This log file is then tracked as a data input to get the data into Splunk and is part of a custom app. The log file has entries that look like the following:

Date: 2020-01-01
Path: /opt/splunk/etc/auth/cert1.pem
Certificate:
    Data:
        Version: 1 (0x0)
Date: 2020-01-01 01:02:03.555
Path: /opt/splunk/etc/auth/cert2.pem
Certificate:
    Data:
        Version: 3 (0x2)
Date: 2020-01-01 01:02:03.555
Path: /opt/splunk/etc/auth/cert3.pem
Certificate:
    Data:
        Version: 3 (0x2)

I have setup a new index and am getting the data into Splunk so that it is searchable but I have not been able to quite get my app's local/props.conf dialed in perfectly. Here is my current props.conf:

#BREAK_ONLY_BEFORE_DATE = true
DATETIME_CONFIG =
LINE_BREAKER = ([\n\r]+)^Date:\s\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}
NO_BINARY_CHECK = true
#SHOULD_LINEMERGE = false
SHOULD_LINEMERGE = true
#BREAK_ONLY_BEFORE = ([\n\r]+)^Date:\s\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}
category = Custom
description = Format custom logfile with decoded PEM certificate information for Splunk servers.
pulldown_type = 1
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 23
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = ^Date:\s
TZ = GMT

The issue I'm seeing right now is that Splunk is breaking the first two lines of each entry in the log file (containing Date and Path) into an event and then the rest of the information starting with Certificate until it hits the next Date entry as another event. 

So for each entry in the log I get the following two events:

Event #1: Date & Path lines

Event #2: All remaining lines starting with Certificate until, but not including, the next occurrence of Date

Ideally, I would like Splunk to have events start at Date and end at the line before the next occurrence of Date. The weird thing is with the current props.conf configuration, I did get one event to be parsed correctly. I am continuing to modify this and will report back if I do resolve this, but any help in parallel is appreciated. This is my first time going through this exercise. Thanks!

[tscroggins]

Hi,

You're very close!

Set SHOULD_LINEMERGE = false in your props.conf stanza.

The caret (^) in your regular expression is superfluous in multiline mode but will break the regular expression in single-line mode. For compatibility irrespective of Splunk's default flags, I would remove it:

LINE_BREAKER = ([\n\r]+)Date:\s\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}

[g9s0x1 · Accepted solution]

Thank you for the tips. I got rid of the ^ in LINE_BREAKER and TIME_PREFIX and set SHOULD_LINEMERGE=false. That almost worked, and I will say it cleaned up how the double events were displaying but I was still getting two events with one that had Date and Path, and one that had everything after Certificate up until the next occurrence of Date.

Looking at the Splunk docs for props.conf in a little more detail, I ended up setting SHOULD_LINEMERGE=true. This ended up working. Splunk docs say "When you set this to 'true', Splunk software combines several lines of data into a single multi-line event." Since I was having issues with the intended singular event being split into two and it is a multi-line event, I went with this setting. I restarted Splunk, re-ran my bash script, and re-ran my search and now the events are being broken up properly at each occurrence of "Date:" within the log.

My final props.conf for reference. Thank you!

DATETIME_CONFIG =
LINE_BREAKER = ([\n\r]+)Date:\s\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
category = Custom
description = Format custom logfile with decoded PEM certificate information for Splunk servers.
pulldown_type = 1
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 23
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = Date:\s
TZ = GMT