Getting Data In

Line break involving CRLF

Explorer

Hi,

I have JSON msgs in my log which has Carriage Return Line Feed character at the end of each line and the next line.

It looks as follows..

Line 1: "Message 1......." [CRLF]
Line 2: [CRLF]
Line 3 : " message 2......"[ CRLF]
Line 4: [CRLF]
Line 5: " Message 3 .... " [ CRLF]

I've configured my props.conf as
[sourcetype]
SHOULD_LINEMERGE = false
LINE_BREAKER = [\r\n]+
KV_MODE= json
N0_BINARY_CHECK=1

If I view my logs from UI, few of my events are single lines and others are combination of two lines, three lines or even more. Can some one help me fix this. What is the regex we must use to match CRLF at the end of line and in the next line ( this line is empty)

Thanks

Tags (1)
0 Karma

Splunk Employee
Splunk Employee

Hi,

The LINE_BREAKER regex requires a capture group to tell Splunk what to discard. Have you tried this as your regular expression?

[sourcetype]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)

Regards

0 Karma

Explorer

Yes jsie, I did add the capture group in my regex. It didn't work

0 Karma

Splunk Employee
Splunk Employee

Can you share a sanitized sample of the raw data? Either include it here within as a code element, or via pastebin (or similar).

0 Karma

Explorer

1) ##SPlunkFileHeader## ************************* LF
2) {"name": *****Json msg***** "} Destination= ' < value of field from header> ' LF
3) CR LF
4) { "client-name":******* ,Json msg>******"} Destination = ' Value' LF
5) CR LF
.....

There is a LF after every line and the next line has CR LF

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!