topic Re: Unable to get substring from the splunk log in Splunk Search

Unable to get substring from the splunk log

karthi25 — Mon, 23 Mar 2020 13:32:32 GMT

I have an requirement to get only the exception related substring from the splunk log,
My log will be in the following format:

fetching records from AAA table
creating event to send to sqs
Publishing to SQS
Large-payload support enabled.
Exception occurred while processing rules for Feed name AAA. Skipping Exception
com.amazonaws.services.sqs.model.QueueDoesNotExistException: The specified queue does not exist for this wsdl version. (Service: AmazonSQS; Status Code: 400; Error Code: AWS.SimpleQueueService.NonExistentQueue; Request ID: xxxx)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1640)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
    .....

Now I want to get only the part of exception from above log like

Exception occurred while processing rules for Feed name AAA. Skipping Exception com.amazonaws.services.sqs.model.QueueDoesNotExistException

I have tried the below query :

index=***  source=*** *Exception* | rex field=_raw "\(Exception occurred while processing rules for Feed name (?<myField>[^\)]:*)\)\("
 | table myField

But it returns empty result. Can anyone please suggest me the right solution for it.

Re: Unable to get substring from the splunk log

gcusello — Mon, 23 Mar 2020 13:53:11 GMT

Hi @karthi25,
if you want all the message:

Exception occurred while processing rules for Feed name AAA. Skipping Exception com.amazonaws.services.sqs.model.QueueDoesNotExistException

try this regex

 (?ms)(?<my_field>Exception occurred while processing rules for [^:]+)

that you can test at https://regex101.com/r/qotuBa/1

If instead, you fon't want the first part of the message, you have to move the part that you don't want in my_field before the parenthesis

(?ms)Exception occurred while processing rules for (?<my_field>[^:]+)

Ciao.
Giuseppe

Re: Unable to get substring from the splunk log

karthi25 — Mon, 23 Mar 2020 14:00:09 GMT

@gcusello Thanks for your reply. I have tried yours , it's throwing me error as below
Error in 'SearchParser': Missing a search command before '('. Error at position '58' of search query 'search index=*** source=*** Exce...{snipped} {errorcontext = ception |(?ms)(?

Re: Unable to get substring from the splunk log

gcusello — Mon, 23 Mar 2020 14:16:34 GMT

Hi @karthi25,
what'ss the search you used?

Try this:

index=***  source=*** *Exception* 
| rex field=_raw "(?ms)(?<my_field>Exception occurred while processing rules for [^:]+)"
| table myField

Ciao.
Giuseppe

Re: Unable to get substring from the splunk log

karthi25 — Mon, 23 Mar 2020 14:37:51 GMT

@gcusello Same only.

Re: Unable to get substring from the splunk log

gcusello — Mon, 23 Mar 2020 14:57:05 GMT

Hi @karthi25,
where are you using this search: in Search form or in a dashboard?
try it in the Search form.

If you want to use it in a dashboard you have to modify the <> chars in:

"<" becames "&lt;"
">" becames "&gt;"

Ciao.
Giuseppe

Re: Unable to get substring from the splunk log

karthi25 — Mon, 23 Mar 2020 15:54:50 GMT

@gcusello Am using in search for creating alert.

Re: Unable to get substring from the splunk log

gcusello — Mon, 23 Mar 2020 16:09:08 GMT

Hi @karthi25,
it seems the you didn't used the rex command

index=*  source=* Exception 
| rex "(?ms)(?<my_field>Exception occurred while processing rules for [^:]+)"
| table my_field

Please try again.

And, please, to display code (as regexes or searches) please use the Code Sample button (the one with 101010).

Ciao.
Giuseppe

Re: Unable to get substring from the splunk log

to4kawa — Mon, 23 Mar 2020 18:59:00 GMT

REGEX:

(?ims)(?<exception>(exception).*\2)

But your original REGEX

 "\(Exception occurred while processing rules for Feed name (?<myField>[^\)]:*)\)\("

what's \(?

your provided log is not with (Exception occurred ...