topic Re: Parsing Multiline logs into single event in Getting Data In

Parsing Multiline logs into single event

Dingu — Wed, 17 Jun 2020 22:56:30 GMT

Hello All,

I'm new to splunk and working on one of my error logs onboarding to splunk. somehow my props.conf for the below sample logs seems to be not working. Any help could be greatly appreciated.

2020-06-05 07:03:56,639
2020-06-05 11:03:56,639
ERROR
[11]
Error Message: [RecId=xxxxxxxx][host=xxxxxxx][userID=xxxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Re: Parsing Multiline logs into single event

richgalloway — Thu, 18 Jun 2020 00:35:02 GMT

Please share your current props.conf settings and show where the event(s) should break.

Re: Parsing Multiline logs into single event

Dingu — Thu, 18 Jun 2020 14:30:35 GMT

@richgalloway Thank you for your response. I have set my current props.conf as follow

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto

And I would like to break my event breaking at the highlighted part.

--------------------------------------------------------------------------------------------------------------------------------------
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Re: Parsing Multiline logs into single event

richgalloway — Thu, 18 Jun 2020 14:42:59 GMT

I think you just need to add a LINE_BREAKER setting.

[sourcetype] TRUNCATE=240000 SHOULD_LINEMERGE=false TIME_PREFIX=^ TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N MAX_TIMESTAMP_LOOKAHEAD=30 KV_MODE = auto LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d[\r\n]+\d\d\d\d

The goal is to break lines before the first timestamp and not also before the second timestamp.