Partially eliminate error message in Splunk query

[Dharani]

Hi,

below are the log details.

index=ABC sourcetype=logging_0

Below are the values of "ErrorMessages" field:

invalid - 5 count

unprocessable - 7 count (5 invalid pair + 2 others)

no user foundv- 3 count

invalid message process - 3 count

process failed- 3 count

Now I have to eliminate ErrorMessage=invalid and ErrorMessage=unprocessable. Then show all other  ErrorMessage.

But the problem here is , "unprocessable" ErrorMessage will show for other messages as well. so we cannot fully eliminate the "unprocessable" ErrorMessage.

Whenever "Invalid" ErrorMessage is logging that time "unprocessable" ErrorMessage also will be logged. So we need to eliminate this pair only. Not every "unprocessable" ErrorMessage.

Expected result:

unprocessable - 2 count

no user foundv- 3 count

invalid message process - 3 count

process failed- 3 count

I tried with join using requestId but its not resulting anything because i am using

| search ErrorMessage="Invalid" and elimated this in next query so its not searching for other ErrorMessages.

Can someone please help.

[yuanliu]

Based on your latest update, the problem should be restated as: remove events with requestId that has a corresponding ValidationErrors value of "Invalid product". (I assume that the trailing space in sample data is a typo.) Is this correct?

In the format illustrated in sample data, Splunk should have given you compliant JSON in ValidationErrors.  Process this first, then literally implement the restated objective.

| spath input=response
| stats values(*) as * by sessionId request requestId responseStatus
| where NOT ValidationErrors == "Invalid product"

Your sample data will leave you with

sessionId	request	requestId	responseStatus	DeveloperMessage	DocumentationUrl	ErrorCode	LogId	Parameters	UserMessage	ValidationErrors
855762c0-9a6b	PUT	bc819b42-6655	422

This is the emulation used to test the method:

| makeresults
| fields - _time
| eval data = mvappend("IBroker call failed, sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT  responseStatus=422  response={\"ErrorCode\":0,\"UserMessage\":null,\"DeveloperMessage\":null,\"DocumentationUrl\":null,\"LogId\":null,\"ValidationErrors\":\"Invalid product\",\"Parameters\":null}",
"sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422  ErrorMessage: unprocessable",
"sessionId=855762c0-9a6b, requestId=bc819b42-6655, request=PUT  responseStatus=422 ErrorMessage: unprocessable")
| mvexpand data
| rename data AS _raw
| extract
``` data emulation above ```

[VatsalJagani]

@Dharani - I think you explained the question well, but you need to provide sample logs to explain what do you mean by pair of error events.

[Dharani]

sample logs:

1.IBroker call failed, sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT  responseStatus=422  response={"ErrorCode":0,"UserMessage":null,"DeveloperMessage":null,"DocumentationUrl":null,"LogId":null,"ValidationErrors":"Invalid product ","Parameters":null}

2. sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT responseStatus=422  ErrorMessage: unprocessable

3.sessionId=855762c0-9a6b, requestId=bc819b42-6646, request=PUT  responseStatus=422 ErrorMessage: unprocessable

1st 2 logs should be eliminated because they share same requestId, 3 rd logs should be shown.

[VatsalJagani]

@Dharani - Do you want to see only the last event per RequestId? (like only the latest error per request is right info?)

[Dharani]

Yes , sorry for the typo. 

3rd logs has different requestId. I mistakenly pasted the same requestId.

[VatsalJagani]

@Dharani - Try response by @yuanliu