Getting Data In
×

Are you a member of the Splunk Community?

Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Ask a Question
Solved! Jump to solution

Parsing Multiline logs into single event

Dingu
Explorer
‎06-17-2020 03:56 PM

Hello All,

I'm new to splunk and working on one of my error logs onboarding to splunk. somehow my props.conf for the below sample logs seems to be not working. Any help could be greatly appreciated.

2020-06-05 07:03:56,639
2020-06-05 11:03:56,639
ERROR
[11]
Error Message: [RecId=xxxxxxxx][host=xxxxxxx][userID=xxxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Labels (1)
Labels
0 Karma
Reply
1 Solution
Solved! Jump to solution
Solution

richgalloway
SplunkTrust
SplunkTrust
‎06-18-2020 07:42 AM

I think you just need to add a LINE_BREAKER setting.

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto
LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d[\r\n]+\d\d\d\d

The goal is to break lines before the first timestamp and not also before the second timestamp.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

Reply
Solved! Jump to solution

richgalloway
SplunkTrust
SplunkTrust
‎06-17-2020 05:35 PM
Please share your current props.conf settings and show where the event(s) should break.
---
If this reply helps you, Karma would be appreciated.
Reply
Solved! Jump to solution

Dingu
Explorer
‎06-18-2020 07:30 AM

@richgalloway Thank you for your response. I have set my current props.conf as follow

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto

And I would like to break my event breaking at the highlighted part.

2020-06-05 07:03:56,639
2020-06-05 11:03:56,639
ERROR
[11]
Error Message: [RecId=xxxxxxxx][host=xxxxxxx][userID=xxxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

--------------------------------------------------------------------------------------------------------------------------------------
2020-06-05 07:03:56,843
2020-06-05 11:03:56,843
ERROR
[15]
Error Message: [RecId=xxxxxxxx][host=xxxxxx][userID=xxxxxx] Exception: System.InvalidOperationException: The view 'Error' or its master was not found or no view engine supports the searched locations. The following locations were searched:
~/Areas/Summary/Views/PaySummary/Error.aspx
~/Areas/Summary/Views/PaySummary/Error.ascx
~/Areas/Summary/Views/Shared/Error.aspx
~/Areas/Summary/Views/Shared/Error.ascx
~/Views/PaySummary/Error.aspx
~/Views/PaySummary/Error.ascx
~/Views/Shared/Error.aspx
~/Views/Shared/Error.ascx
~/Areas/Summary/Views/PaySummary/Error.cshtml
~/Areas/Summary/Views/PaySummary/Error.vbhtml
~/Areas/Summary/Views/Shared/Error.cshtml
~/Areas/Summary/Views/Shared/Error.vbhtml
~/Views/PaySummary/Error.cshtml
~/Views/PaySummary/Error.vbhtml
~/Views/Shared/Error.cshtml
~/Views/Shared/Error.vbhtml
at System.Web.Mvc.ViewResult.FindView(ControllerContext context)
at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__5(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult)
at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

0 Karma
Reply
Solved! Jump to solution
Solution

richgalloway
SplunkTrust
SplunkTrust
‎06-18-2020 07:42 AM

I think you just need to add a LINE_BREAKER setting.

[sourcetype]
TRUNCATE=240000
SHOULD_LINEMERGE=false
TIME_PREFIX=^
TIME_FORMAT= %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD=30
KV_MODE = auto
LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d[\r\n]+\d\d\d\d

The goal is to break lines before the first timestamp and not also before the second timestamp.

---
If this reply helps you, Karma would be appreciated.
Reply
Get Updates on the Splunk Community!

What the End of Support for Splunk Add-on Builder Means for You

Hello Splunk Community! We want to share an important update regarding the future of the Splunk Add-on Builder ...

Solve, Learn, Repeat: New Puzzle Channel Now Live

Welcome to the Splunk Puzzle PlaygroundIf you are anything like me, you love to solve problems, and what ...

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...