topic Re: how to remove multiple logs into single event in Getting Data In

how to remove multiple logs into single event

duggp007 — Wed, 30 Sep 2020 02:31:11 GMT

[tomcat]
EXTRACT = \/u01\/logs-(?\w+)\/.* in source

Adding the below to BREAK EVENTS only at timestamp and TRUNCATING issue

BREAK_ONLY_BEFORE = (\d+[- :,-w]+)
MAX_TIMESTAMP_LOOKAHEAD = 30
TRUNCATE = 0

We are facing an issue with multiple logs in a single event for only tomcat as the sourcetype,

May I know the reason for it.
we also have
SHOULD_LINEMERGE=true for other sourcetype should I include SHOULD_LINEMERGE=false for the tomcat.

Any help will be appreciated.

Re: how to remove multiple logs into single event

richgalloway — Thu, 10 Oct 2019 20:32:17 GMT

Your line breaking settings in props.conf are wrong. We'll need to see some sample data to give the best settings.

Re: how to remove multiple logs into single event

gcusello — Fri, 11 Oct 2019 10:00:24 GMT

HI duggp007,
surely you have to review your parsing.
To do this I suggest to take an extract of your logs with all the kind of logs in the same sourcetype (e.g. tomcat) in a text file.
Then ingest it for test using the guided procedure in web interface [Settings -- Add Data] so you can find the correct options yo use for thst sourcetype.
Probably is uncorrect the TIME_PREFIX and/or the TIME_FORMAT, so, sometimes, Splunk cannot recognize the start of an event and doesn't correctly break it.

Bye.
Giuseppe

Re: how to remove multiple logs into single event

woodcock — Sun, 13 Oct 2019 08:59:35 GMT

You should always avoid the BREAK_* settings and use only SHOULD_LINEMERGE=false and LINE_BREAKER = Your RegEx Here.