Getting Data In

log file not parsing properly - multiple lines per event

dglass0215
Path Finder

Hello,

I have a file monitor for a log file where I am getting indexed data with multiple lines. Example of one event:

2019-12-30 09:16:41:908: Requestor: IMM_Mobile, IsLocal: False
2019-12-30 09:16:41:908: 637132942019089151: Scanned CID: BARCODE:

Now i notice that it is the same time but they should still be separate events. i have read where someone suggested SHOULD_LINEMERGE = false, however if I am reading the documentation correctly, the SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true being the defaults should be processing the above as two separate events. What am I misunderstanding?

I am hesitant to configure SHOULD_LINEMERGE = false because I think it may be needed for other events that span multiple lines.

only other thing I can think of is possibly my props/transforms might be screwing with the data in some other way. Below are what I think are the relevant portions of my props/transforms:

Props:
[mySourceType]
TRANSFORMS-set= setnull,setparsing
TRANSFORMS-sourcetype= setNewSourceType

Transforms:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \b(?:offline|online|\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)\b
DESK_KEY = queue
FORMAT = indexQueue

[setNewSourceType]
REGEX = \b(\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)
FORMAT = sourcetype::NewSourceType
DEST_KEY = MetaData:Sourcetype

Thanks for any assistance!
David

0 Karma

mydog8it
Builder

Take a look at this data onboarding cheat sheet published by Aplura, it is a great guide to establish your data onboarding practices:
https://www.aplura.com/assets/pdf/onboarding_cheatsheet.pdf

0 Karma

dglass0215
Path Finder

While this might be slightly useful it does not answer my question. Do you know why the two lines above are one event? And do you know how I can fix it? Thanks!

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...