topic Partially eliminate error message in Splunk query in Splunk Search

Partially eliminate error message in Splunk query

Dharani — Thu, 14 Dec 2023 08:54:15 GMT

Hi,

below are the log details.

index=ABC sourcetype=logging_0

Below are the values of "ErrorMessages" field:

invalid - 5 count

unprocessable - 7 count (5 invalid pair + 2 others)

no user foundv- 3 count

invalid message process - 3 count

process failed- 3 count

Now I have to eliminate ErrorMessage=invalid and ErrorMessage=unprocessable. Then show all other ErrorMessage.

But the problem here is , "unprocessable" ErrorMessage will show for other messages as well. so we cannot fully eliminate the "unprocessable" ErrorMessage.

Whenever "Invalid" ErrorMessage is logging that time "unprocessable" ErrorMessage also will be logged. So we need to eliminate this pair only. Not every "unprocessable" ErrorMessage.

Expected result:

unprocessable - 2 count

no user foundv- 3 count

invalid message process - 3 count

process failed- 3 count

I tried with join using requestId but its not resulting anything because i am using

| search ErrorMessage="Invalid" and elimated this in next query so its not searching for other ErrorMessages.

Can someone please help.

Re: Partially eliminate error message in Splunk query

VatsalJagani — Thu, 14 Dec 2023 12:39:10 GMT

@Dharani - I think you explained the question well, but you need to provide sample logs to explain what do you mean by pair of error events.

Re: Partially eliminate error message in Splunk query

Dharani — Fri, 15 Dec 2023 07:40:23 GMT

sample logs:

1.IBroker call failed, sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422 response={"ErrorCode":0,"UserMessage":null,"DeveloperMessage":null,"DocumentationUrl":null,"LogId":null,"ValidationErrors":"Invalid product ","Parameters":null}

2. sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422 ErrorMessage: unprocessable

3.sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422 ErrorMessage: unprocessable

1st 2 logs should be eliminated because they share same requestId, 3 rd logs should be shown.

Re: Partially eliminate error message in Splunk query

VatsalJagani — Fri, 15 Dec 2023 08:25:15 GMT

@Dharani - Do you want to see only the last event per RequestId? (like only the latest error per request is right info?)

Re: Partially eliminate error message in Splunk query

Dharani — Fri, 15 Dec 2023 09:55:44 GMT

Yes , sorry for the typo.

3rd logs has different requestId. I mistakenly pasted the same requestId.

Re: Partially eliminate error message in Splunk query

yuanliu — Fri, 15 Dec 2023 10:56:16 GMT

Based on your latest update, the problem should be restated as: remove events with requestId that has a corresponding ValidationErrors value of "Invalid product". (I assume that the trailing space in sample data is a typo.) Is this correct?

In the format illustrated in sample data, Splunk should have given you compliant JSON in ValidationErrors. Process this first, then literally implement the restated objective.

| spath input=response | stats values(*) as * by sessionId request requestId responseStatus | where NOT ValidationErrors == "Invalid product"

Your sample data will leave you with

sessionId	request	requestId	responseStatus	DeveloperMessage	DocumentationUrl	ErrorCode	LogId	Parameters	UserMessage	ValidationErrors
855762c0-9a6b	PUT	bc819b42-6655	422

This is the emulation used to test the method:

| makeresults | fields - _time | eval data = mvappend("IBroker call failed, sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422 response={\"ErrorCode\":0,\"UserMessage\":null,\"DeveloperMessage\":null,\"DocumentationUrl\":null,\"LogId\":null,\"ValidationErrors\":\"Invalid product\",\"Parameters\":null}", "sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422 ErrorMessage: unprocessable", "sessionId=855762c0-9a6b, requestId=bc819b42-6655, request=PUT responseStatus=422 ErrorMessage: unprocessable") | mvexpand data | rename data AS _raw | extract ``` data emulation above ```

Re: Partially eliminate error message in Splunk query

VatsalJagani — Mon, 18 Dec 2023 10:38:07 GMT

@Dharani - Try response by @yuanliu