Getting Data In

How to merge multiline messages to one event

akanno
Communicator

Hi,splunkers

We want to index multiline log messages with no timestamp as one event.

But regular expression for multiline is difficult.

So now I try following configurations.

[source::/opt/mail1.log]

SHOULD_LINEMERGE = true

MAX_EVENTS=200

LINE_BREAKER = XXXXXXXXXXXXX

TRUNCATE = 50000

But it does not work.

first event is 200 lines messages event but next event is 1 line messages event.

I want to 200 lines messages per one event.

Is there any idea?

thank you for my help,

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

View solution in original post

lguinn2
Legend

Try this

[source::/opt/mail1.log]
SHOULD_LINEMERGE = true
MAX_EVENTS=200
BREAK_ONLY_BEFORE=XXXXXXXXXXXX
TRUNCATE = 50000
DATETIME_CONFIG = NONE

akanno
Communicator

Sorry,
Mistake made by me.
This answer is good.
Thank you very much for lguinn ♦ .

0 Karma

akanno
Communicator

inputs.conf is on universal forwarder for this input
props.conf is on indexer

0 Karma

lguinn2
Legend

So both your inputs.conf and your props.conf are on the indexer for this input?

0 Karma

akanno
Communicator

It is on a indexer

0 Karma

lguinn2
Legend

Next question: where is your input? Is it on a forwarder? A universal forwarder or a heavy forwarder?

Where is your props.conf?

0 Karma

akanno
Communicator

I tried this answer but I had the same result

0 Karma

akanno
Communicator

Hi,lguinn ♦ thank you for comment
Assumed log is continuous with same messages
For example
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP
Rejected at IN(default) filter: TCP

0 Karma

lguinn2
Legend

We need to see an example of your data. This is not enough information!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...