Getting Data In

How to merge multiline messages to one event

akanno
Communicator

Hi,splunkers

We want to index multiline log messages with no timestamp as one event.

But regular expression for multiline is difficult.

So now I try following configurations.

[source::/opt/mail1.log]

SHOULD_LINEMERGE = true

MAX_EVENTS=200

LINE_BREAKER = XXXXXXXXXXXXX

TRUNCATE = 50000

But it does not work.

first event is 200 lines messages event but next event is 1 line messages event.

I want to 200 lines messages per one event.

Is there any idea?

thank you for my help,

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

View solution in original post

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

akanno
Communicator

Sorry,
Mistake made by me.
This answer is good.
Thank you very much for lguinn ♦ .

0 Karma

akanno
Communicator

inputs.conf is on universal forwarder for this input
props.conf is on indexer

0 Karma

lguinn2
Legend

So both your inputs.conf and your props.conf are on the indexer for this input?

0 Karma

akanno
Communicator

It is on a indexer

0 Karma

lguinn2
Legend

Next question: where is your input? Is it on a forwarder? A universal forwarder or a heavy forwarder?

Where is your props.conf?

0 Karma

akanno
Communicator

I tried this answer but I had the same result

0 Karma

akanno
Communicator

Hi,lguinn ♦ thank you for comment
Assumed log is continuous with same messages
For example
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP

0 Karma

lguinn2
Legend

We need to see an example of your data. This is not enough information!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...