Getting Data In
×

Join the Conversation

Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
Ask a Question
Solved! Jump to solution

Parsing Multiline logs into single event

Dingu
Explorer
‎06-17-2020 03:56 PM

Hello All,

I'm new to splunk and working on one of my error logs onboarding to splunk. somehow my props.conf for the below sample logs seems to be not working. Any help could be greatly appreciated.

2020-06-05 07:03:56,639
2020-06-05 11:03:56,639
ERROR
[11]
Error Message: [RecId=xxxxxxxx][host=xxxxxxx][userID=xxxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Labels (1)
Labels
0 Karma
Reply
1 Solution
Solved! Jump to solution
Solution

richgalloway
SplunkTrust
SplunkTrust
‎06-18-2020 07:42 AM

I think you just need to add a LINE_BREAKER setting.

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto
LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d[\r\n]+\d\d\d\d

The goal is to break lines before the first timestamp and not also before the second timestamp.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

Reply
Solved! Jump to solution

richgalloway
SplunkTrust
SplunkTrust
‎06-17-2020 05:35 PM
Please share your current props.conf settings and show where the event(s) should break.
---
If this reply helps you, Karma would be appreciated.
Reply
Solved! Jump to solution

Dingu
Explorer
‎06-18-2020 07:30 AM

@richgalloway Thank you for your response. I have set my current props.conf as follow

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto

And I would like to break my event breaking at the highlighted part.

2020-06-05 07:03:56,639
2020-06-05 11:03:56,639
ERROR
[11]
Error Message: [RecId=xxxxxxxx][host=xxxxxxx][userID=xxxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

--------------------------------------------------------------------------------------------------------------------------------------
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

0 Karma
Reply
Solved! Jump to solution
Solution

richgalloway
SplunkTrust
SplunkTrust
‎06-18-2020 07:42 AM

I think you just need to add a LINE_BREAKER setting.

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto
LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d[\r\n]+\d\d\d\d

The goal is to break lines before the first timestamp and not also before the second timestamp.

---
If this reply helps you, Karma would be appreciated.
Reply
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...