Splunk Search

How to fix issue with parsing events (log files)?

SplunkDash
Motivator

Hey,

I have issues with parsing events, multiple events/records (raw data) are within the same event. Sample data and my props configuration file are giving below. How help will be highly appreciated. Thank you so much in advance for your help:

Sample Events

May 92023 5:46:00 AM com.vontu.messaging.chainData.PremiseMessageChainTracer beginChain FINER: Message chain #5: Begin processing message [0C369823455-7843-44D7-89E3-SAB21BF361F24Ffrom [Request].

May 92023 5:46:00 AM com.vontu.messaging.chainData.ComponentProcessor$PerMessageProcessor processMessageComponents FINER: Processing of message [0C369823655-7843-44D7-89E3-B21BF361F24F]:[Unknowntook: 0 ms

May 92023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessor$PerMessageProcessor processMessageComponents FINER: Processing of message [0C369823-7843-44D7-89E3-B21BF361F24F]:[Unknowntook: 0 ms

May 92023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessor$PerMessageProcessor processMessageComponents FINER: Processing of message [0C3698sdss23-7843-44D7-89E3-B21BF361F24F]:[attached-email-body.txttook: 11 ms

May 92023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessorr$PerMessageProcessor processMessageComponents FINER: Processing of message [0C3698saaa23-7843-44D7-89E3-B21BF361566F24F]:[Unknowntook: 10 ms

May 92023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessor$PerMessageProcessor processMessageComponents FINER: Processing of message [0C3698sdaa23-7843-44D7-89E3-B21BF361F24F]:[[EXT] [LibraryLinkLibrary Link of the Day for 2023-05-09_attached-email-bodytook: 9 ms

May 92023 5:46:00 AM com.vontu.messaging.chain.imagepreclassifier.ImagePreclassifierManager applyPrefiltersOnImages INFO: Skipping component: unknown for image filtering as required component.

 

PROPS.CONF

[auditrdata]

SHOULD_LINEMERGE=false

LINE_BREAKER=([\r\n]+)\w+\s\d{2},\s\d{4}

NO_BINARY_CHECK=true

CHARSET=UTF-8

disabled=false

TIME_PREFIX=^

TIME_FORMAT=%b %d, %Y %H:%M:%S

MAX_TIMESTAMP_LOOKAHEAD=30

TRUNCATE=5000

Labels (1)
Tags (1)
0 Karma
1 Solution

m_pham
Splunk Employee
Splunk Employee

Adding to @VatsalJagani 's suggestion, try this:

[auditrdata]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\w+\s\d{1,2},\s\d{4}
NO_BINARY_CHECK=true
TIME_PREFIX=^
TIME_FORMAT=%b %d, %Y %I:%M:%S %p
MAX_TIMESTAMP_LOOKAHEAD=30
TRUNCATE=5000

  

View solution in original post

goncalocoelho
Path Finder

Hi, 

the problem is in Line Breaker

LINE_BREAKER=([\r\n]+)\w+\s\d{1,2},\s\d{4}

 

VatsalJagani
SplunkTrust
SplunkTrust

@SplunkDash - I hope the sourcetype is correct as you said, so try the below configuration:

 

[auditrdata]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\w+\s\d{1,2},\s\d{4}
NO_BINARY_CHECK=true
TIME_PREFIX=^
TIME_FORMAT=%b %d, %Y %I:%M:%S %p
MAX_TIMESTAMP_LOOKAHEAD=30
TRUNCATE=5000

 

 

I hope this helps!!! Kindly upvote if it does!!!

SplunkDash
Motivator

Hello @VatsalJagani ,

Thank you so much for your quick response, truly appreciate it. Now it's parsing one event as 2 events. I think the sample events I provided you should have 2 lines for each event, but the way I copied it looked like one line sorry about that. Each of the event should be as follow. Any recommendations would be highly appreciated. Thank you so much again.

May 9, 2023 5:46:00 AM com.vontu.messaging.chainData.PremiseMessageChainTracer beginChain 
FINER: Message chain #5: Begin processing message [0C369823455-7843-44D7-89E3-SAB21BF361F24F] from [Request].

May 9, 2023 5:46:00 AM com.vontu.messaging.chainData.ComponentProcessor PerMessageProcessor processMessageComponents 
FINER: Processing of message [0C369823655-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

May 9, 2023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessorPerMessageProcessor processMessageComponents
FINER: Processing of message [0C369823-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

 

 

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@SplunkDash - Just updated my original response based on the change that you asked.

Just updated the LINE_BREAKER to 

([\r\n]+)\w+\s\d{1,2},\s\d{4}

 

Try original response now.

0 Karma

m_pham
Splunk Employee
Splunk Employee

Adding to @VatsalJagani 's suggestion, try this:

[auditrdata]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\w+\s\d{1,2},\s\d{4}
NO_BINARY_CHECK=true
TIME_PREFIX=^
TIME_FORMAT=%b %d, %Y %I:%M:%S %p
MAX_TIMESTAMP_LOOKAHEAD=30
TRUNCATE=5000

  

SplunkDash
Motivator

Hello @VatsalJagani

@m_pham 

@goncalocoelho : 

Thank you so much again. Now events are parsing without doubling up (Multiple) events within single event. But now issue with the Line that has the TIMESTAMP. Every event is missing Line that has the TIMESTAMP and showing as follow. Any help will be highly appreciated, thank you again.

Showing Now:

FINER: Message chain #5: Begin processing message [0C369823455-7843-44D7-89E3-SAB21BF361F24F] from [Request].

FINER: Processing of message [0C369823655-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

FINER: Processing of message [0C369823-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

Should be:

May 9, 2023 5:46:00 AM com.vontu.messaging.chainData.PremiseMessageChainTracer beginChain 
FINER: Message chain #5: Begin processing message [0C369823455-7843-44D7-89E3-SAB21BF361F24F] from [Request].

May 9, 2023 5:46:00 AM com.vontu.messaging.chainData.ComponentProcessor PerMessageProcessor processMessageComponents 
FINER: Processing of message [0C369823655-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

May 9, 2023 5:46:00 AM com.vontu.messaging.chain.ComponentProcessorPerMessageProcessor processMessageComponents
FINER: Processing of message [0C369823-7843-44D7-89E3-B21BF361F24F]:[Unknown] took: 0 ms

0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@SplunkDash - Try btool and show config CLI command to see what configuration is placed for this sourcetype and there is no conflicting configuration already present in your Splunk environment.

0 Karma

SplunkDash
Motivator

Hello @VatsalJagani , @goncalocoelho , @m_pham 

Thank you so much again, this is completely new ingestion and no conflict found. Now is one interesting thing here, getting some of the events with the proper structure (with TIMESTAMP Line or no missing line) and some other events without that, thinking there might be issues (or inconsistency) with the format of the TIMESTAMP causing that issue. What you think? If this is the issue, what should I do, any recommendation would be highly  appreciated.

Tags (1)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@SplunkDash - The recommendation is to first find the different format that is causing the issue (or find all the different formats) and then based on that we maybe able to suggest something.

* Also it is unusual for a single system to generate two different timestamp formats for the same data.

* It could be either a different host, or different source.

SplunkDash
Motivator

@VatsalJagani@goncalocoelho@m_pham 

Thank you so much you all, truly appreciate it. Let me try with this and let you know how it goes. Thank you so much again.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...