Getting Data In

log file not parsing properly - multiple lines per event

dglass0215
Path Finder

Hello,

I have a file monitor for a log file where I am getting indexed data with multiple lines. Example of one event:

2019-12-30 09:16:41:908: Requestor: IMM_Mobile, IsLocal: False
2019-12-30 09:16:41:908: 637132942019089151: Scanned CID: BARCODE:

Now i notice that it is the same time but they should still be separate events. i have read where someone suggested SHOULD_LINEMERGE = false, however if I am reading the documentation correctly, the SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true being the defaults should be processing the above as two separate events. What am I misunderstanding?

I am hesitant to configure SHOULD_LINEMERGE = false because I think it may be needed for other events that span multiple lines.

only other thing I can think of is possibly my props/transforms might be screwing with the data in some other way. Below are what I think are the relevant portions of my props/transforms:

Props:
[mySourceType]
TRANSFORMS-set= setnull,setparsing
TRANSFORMS-sourcetype= setNewSourceType

Transforms:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \b(?:offline|online|\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)\b
DESK_KEY = queue
FORMAT = indexQueue

[setNewSourceType]
REGEX = \b(\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)
FORMAT = sourcetype::NewSourceType
DEST_KEY = MetaData:Sourcetype

Thanks for any assistance!
David

0 Karma

mydog8it
Builder

Take a look at this data onboarding cheat sheet published by Aplura, it is a great guide to establish your data onboarding practices:
https://www.aplura.com/assets/pdf/onboarding_cheatsheet.pdf

0 Karma

dglass0215
Path Finder

While this might be slightly useful it does not answer my question. Do you know why the two lines above are one event? And do you know how I can fix it? Thanks!

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...