I am trying to build a splunk query to get the error summary from a log. I want to capture all the events where there is some ERROR, Exception or Failure.
Below is the sample data :
ERROR org.mule.component.ComponentException: Failed to invoke ScriptComponent{bapmFlow.component.797791858}. Component that caused exception is: ScriptComponent{bapmFlow.component.797791858}.
host = host1 = /odt/mule_/logs/bapm.logsourcetype = gdt_index
2/7/21
12:00:04.000 AM
2021-02-07 00:00:04,422 [[Java2python].bapmFlow.stage1.03] ERROR org.mule.exception.CatchMessagingExceptionStrategy - Failed to dispatch message to error queue after it failed to process. This may cause message loss. Message identification summary here: id=54972f10-6901-11eb-ad2a-0050568f5886 correlationId=<not set>, correlationGroup=-1, correlationSeq=-1
host = host1 = /odt/mule_/logs/bapm.logsourcetype = gdt_index
2021-02-07 00:00:04,407 [[Java2python].bapmFlow.stage1.03] ERROR org.mule.exception.CatchMessagingExceptionStrategy -
********************************************************************************
Message : org.mule.module.db.internal.domain.connection.ConnectionCreationException: Cannot get connection for URL jdbc:sqlserver://VLTROUXRPT.us.global.crux.com\PRS:1713;databaseName=DFT;domain=US;integratedSecurity=false;authenticationScheme=JavaKerberos;userName=Jack;password=<<credentials>>;trustServerCertificate=true;encrypt=true; : Login failed for user 'Jack'. ClientConnectionId:34edad77-7de1-4d0f-bc13-0fb7f090f722 (java.sql.SQLException)
2021-02-07 00:00:02,936 [[Java2python].bapmFlow.stage1.03] ERROR org.mule.exception.CatchMessagingExceptionStrategy -
... 89 lines omitted ...
2021-02-07 00:00:02,951 [[Java2python].bapmFlow.stage1.03] ERROR org.mule.exception.CatchMessagingExceptionStrategy - Failed to dispatch message to error queue after it failed to process. This may cause message loss. Message identification summary here: id=54970800-6901-11eb-a3d3-0050568f5165 correlationId=<not set>, correlationGroup=-1, correlationSeq=-1
I have noticed the below: The ERROR keyword before the failures with the exception name. So I built this basic query like below but it's not giving the desired results:
index=hdt sourcetype=gdt_index ("ERROR" AND "Exception") OR "FAILED" | rex ".*?(?<Exception>(\w+\.)+\w*Exception).*" | rex "(?<ErrorMessage>\"Message\":(.*\",))" | stats values(ErrorMessage) as ErrorMessage by Exception
You can get both values from one rex expression - I extended the Exception part to include other words such as Strategy to get the complete name of the exception, then skip over the non-words (spaces, colons, etc.), then assume the remainder of the line was the error message you wanted.
| rex ".*?(?<Exception>(\w+\.)+\w*Exception\w*)\W+(?<ErrorMessage>.*)"
Thanks @ITWhisperer Yes ..You are right. I was trying to follow the examples I had in my project.
I want the message of the failures which comes right after the exception
For e.g.
Failed to invoke ScriptComponent{bapmFlow.component.797791858}. Component that caused exception is: ScriptComponent{bapmFlow.component.797791858}.
Cannot get connection for URL jdbc:sqlserver://VLTROUXRPT.us.global.crux.com\PRS:1713;databaseName=DFT;domain=US;integratedSecurity=false;authenticationScheme=JavaKerberos;userName=Jack;password=<<credentials>>;trustServerCertificate=true;encrypt=true; : Login failed for user 'Jack'. ClientConnectionId:34edad77-7de1-4d0f-bc13-0fb7f090f722 (java.sql.SQLException)
I want the exception name and the messages with which it failed.
You can get both values from one rex expression - I extended the Exception part to include other words such as Strategy to get the complete name of the exception, then skip over the non-words (spaces, colons, etc.), then assume the remainder of the line was the error message you wanted.
| rex ".*?(?<Exception>(\w+\.)+\w*Exception\w*)\W+(?<ErrorMessage>.*)"
@ITWhisperer Can you also please explain a bit about it.
".*?(?<Exception>(\w+\.)+\w*Exception\w*)\W+(?<ErrorMessage>.*)"
.*? - not really needed since * means 0 or more so could match anything or nothing
(?<Exception>(\w+\.)+\w*Exception\w*) - first capture group
<Exception> - name of field
(\w+\.)+ - one or more groups of "letters" followed by a . e.g. class in exception class hierarchy
\w*Exception - zero or more "letters" followed by Exception
\w* - zero or more "letters"
Strings which match this are put into the Exception field (assuming the rest of the expression matches)
\W+ - one or more "non-letter" e.g. punctuation and spaces
(?<ErrorMessage>.*) - second capture group
<ErrorMessage> - name of field
.* - zero or more of anything until end of the line
@ITWhisperer Thank you so much!! This will help 🙂
@ITWhisperer --This looks Perfect to me!!
Thanks !!
You don't have anything in your example that contains "Message": so ErrorMessage would not contain anything - what were you expecting it to hold?