Getting Data In

How to merge multiline messages to one event

akanno
Communicator

Hi,splunkers

We want to index multiline log messages with no timestamp as one event.

But regular expression for multiline is difficult.

So now I try following configurations.

[source::/opt/mail1.log]

SHOULD_LINEMERGE = true

MAX_EVENTS=200

LINE_BREAKER = XXXXXXXXXXXXX

TRUNCATE = 50000

But it does not work.

first event is 200 lines messages event but next event is 1 line messages event.

I want to 200 lines messages per one event.

Is there any idea?

thank you for my help,

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

View solution in original post

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

akanno
Communicator

Sorry,
Mistake made by me.
This answer is good.
Thank you very much for lguinn ♦ .

0 Karma

akanno
Communicator

inputs.conf is on universal forwarder for this input
props.conf is on indexer

0 Karma

lguinn2
Legend

So both your inputs.conf and your props.conf are on the indexer for this input?

0 Karma

akanno
Communicator

It is on a indexer

0 Karma

lguinn2
Legend

Next question: where is your input? Is it on a forwarder? A universal forwarder or a heavy forwarder?

Where is your props.conf?

0 Karma

akanno
Communicator

I tried this answer but I had the same result

0 Karma

akanno
Communicator

Hi,lguinn ♦ thank you for comment
Assumed log is continuous with same messages
For example
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP

0 Karma

lguinn2
Legend

We need to see an example of your data. This is not enough information!

0 Karma
Get Updates on the Splunk Community!

Splunk Platform | Upgrading your Splunk Deployment to Python 3.9

Splunk initially announced the removal of Python 2 during the release of Splunk Enterprise 8.0.0, aiming to ...

From Product Design to User Insights: Boosting App Developer Identity on Splunkbase

co-authored by Yiyun Zhu & Dan Hosaka Engaging with the Community at .conf24 At .conf24, we revitalized the ...

Detect and Resolve Issues in a Kubernetes Environment

We’ve gone through common problems one can encounter in a Kubernetes environment, their impacts, and the ...