Getting Data In

How come I'm unable to extract multiline events with the multikv command?

carao2020
New Member

Hi,

I am trying to extract events from multiline event using multikv. Could someone please help me in configuring the multikv for these kind of events?

0 Karma

tiagofbmm
Influencer

The approach should be breaking those events at Indextime, not at search time. You should have your sourcetype with the following parameters:

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=App
TIME_FORMAT=%d/%m/%Y %H:%:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=40
TRUNCATE=1000

That way your events will be well defined, check the image belowalt text

0 Karma

carao2020
New Member

Hi Tiagofbmm,

Thanks for the details. The actual event payload is huge XML file and I had broken down that XML event and tried extracting this event content from XML. Could you please review and suggest if we can split this by multikv conf file.. Thanks in advance.. Additionally could you please suggest whether index time event breaking is good for performance than searchtime event breaking. Thanks in advance

0 Karma

tiagofbmm
Influencer

If the XML is big you can always increase the TRUNCATE parameter to accommodate that. multikv doesn't seem to apply to what you have: every line of your data is not an event. YOur scenario is clearly dependent on proper event breaking settings like I showed you. Can you try using these:

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=App
TIME_FORMAT=%d/%m/%Y %H:%:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=40
TRUNCATE=100000

0 Karma

tiagofbmm
Influencer
0 Karma

carao2020
New Member

Hi Tiagofbmm,

Thanks for the response. Please see below sample event data that I am trying to extract the events using multikv command. I need to create one event for each chunk of data starts with "App - Error:" and Endswith "Solution: " so that I can extract the fields in subsequent stages. Kindly review and suggest configuration for multikv.conf.

<![CDATA[
Sample Software Group, Inc. cerbnav0
Batch Router Module
Version 123.45678.91011.1001 Oct 20 2017 15:41:03
© 2017 Sample Software Group, Inc. All rights reserved. 

Application Server: SAMPLEHOST
Process ID        : 7392
User              : sample_user

*-- Standard Error --*
================================
Initialization Phase Started.

Actual Run Start Date and Time     [01/22/2019 20:09:02.970]

Initialization Phase Complete.
================================
================================
Execution Phase Started.


Sample Software Group, Inc. cmcbclmu
Claims Multi-Eng Electronic Adjud. Program
Version 123.45678.91011.1001 Oct 20 2017 16:00:40
© 2017 Sample Software Group, Inc. All rights reserved. 

Application Server: SAMPLEHOST
Process ID        : 7392
User              : sample_user

*-- Standard Error --*
================================
Initialization Phase Started.

Actual Run Start Date and Time     [01/22/2019 20:09:02.970]

Initialization Phase Complete.
================================
================================
Execution Phase Started.

App - Error: 01/22/2019 20:09:24.530
Return Code: 8
Error Code: 51301
Error Message: ABC697868600 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 710 Warning Text =  COBZ - Other Carrier Paid is Zero  (line 1)
Pend Reason =  COBZ WMUD
Solution:  

App - Error: 01/22/2019 20:11:47.646
Return Code: 8
Error Code: 51301
Error Message: ABCDE4538400 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 710 Warning Text =  MNEC Review Denied Services for Medical Necessity  (line 3)
Pend Reason =  MNEC WMUD
Solution:  

App - Error: 01/22/2019 20:12:20.889
Return Code: 8
Error Code: 51301
Error Message: ABC683235200 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 17 Warning Text =  ATHM Multiple UM pre-authorizations match on line 1
Pend Reason =  ATHM WMWM
Solution:  
0 Karma
Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...