topic Re: Displaying successful and failed results in timechart in Splunk Search

Displaying successful and failed results in timechart?

aasiaa — Mon, 17 Oct 2022 14:30:23 GMT

Hi,

I am trying to show successful validations and failures in one of the dashboard panels. I am logging exceptions in traceData.exception field. In this field there is exception with full stacktrace , but would like to display just exception name so need to extract just first line of exception.

My query looks like

index=xxx sourcetype="xxx" app=xxx event.data.request.uri="/xxx" | rename event.data.response.statusCode as statusCode | rename event.traceData.exception as exception | rex field=exception "(?<exception>.*)" | eval result = if(statusCode=201,"Valid", 'exception') | timechart span=1h count by result

the issue is it displays "Valid" calls, but for exceptions it just displays one exception, and the other one is NULL.

_time Valid Token invalid NULL

2022-10-13 08:00

Both exceptions have the same fields (just different exception values and stacktrace). Could you help me with the query which will display results and extract all the exceptions without stacktrace ?

Re: Displaying successful and failed results in timechart

richgalloway — Sat, 15 Oct 2022 00:09:37 GMT

Please share sanitized samples of validations and failures. We can't check your regular expression without it.

Re: Displaying successful and failed results in timechart

aasiaa — Sat, 15 Oct 2022 08:50:18 GMT

Here are samples of success and failures

Success

app: xxx
   event: {
     data: {
       request: {
       }
       response: {
         statusCode: 201
       }
     }
     traceData: {
       traceId: xxx
     }

Failure 1

app: xxx
   event: {
     data: {
       request: {
       }
       response: {
         statusCode: 401
       }
     }
     traceData: {
       exception: Token inactive
com.xxxx.xxxx.xxx.xxxx.xxxx.xxxx.xxxx.xxxx(xxx:52)
xxxxxxxxxxxxxxxx(xxxxx.xxx:113)

xxxxxxxxxxxxxxxx(xxxxx.xxx:213)
traceId: xxx

}
}

Failure 2

app: xxx
   event: {
     data: {
       request: {
       }
       response: {
         statusCode: 400
       }
     }
     traceData: {
       exception: Id: xxx already used or not found
com.xxxx.xxxx.xxx.xxxx.xxxx.xxxx.xxxx.xxxx(xxx:13)
xxxxxxxxxxxxxxxx(xxxxx.xxx:114)
xxxxxxxxxxxxxxxx(xxxxx.xxx:214)
     traceId: xxx
     }
   }

Re: Displaying successful and failed results in timechart

richgalloway — Sat, 15 Oct 2022 12:25:02 GMT

Thanks for the sample events. Which exception is displayed and which one is not?

The rex command in the query seems meaningless. It looks at the exception field and puts everything it finds into the exception field. Why?

Re: Displaying successful and failed results in timechart

aasiaa — Sat, 15 Oct 2022 13:00:52 GMT

Thank you for trying to help.

It displays second exception and success response. The other one displays as null, but query finds all the events.

Rex I found in someone else's question how to display exception without stacktrace, within all the proposed solution, that was the only one actually working. Without this 'rex' it displays exception with stacktrace.

Re: Displaying successful and failed results in timechart

aasiaa — Sat, 15 Oct 2022 21:50:51 GMT

I have also noticed strange thing about those exceptions not displaying (displaying as NULL). When I search for the events, and in my query use add event.traceData.exception!=NULL or event.traceData.exception=* it does not find them. Also when I click on the exception field and try to add it to a search or exclude it from search it does not do anything. What could cause that ? In the code all those exceptions are logged the same way, the only difference between them is the type of exception. But that really should not matter as I should be able to add to the field whatever I want ...

Re: Displaying successful and failed results in timechart

aasiaa — Sat, 15 Oct 2022 22:17:51 GMT

I have actually found a solution. I found this post fields appearing as null when too long , my exceptions are quite long so it sounded like this is the case with my events, and using solution from there I made my query working.

index=xxx sourcetype="xxx" app=xxx event.data.request.uri="xxx"
| rename event.data.response.statusCode as statusCode
| spath input=_raw path=event.traceData.exception output=exception
| rex field=exception "(?<exception>.*)"
| eval result = if(statusCode=201,"Valid", exception)
| timechart span=1h count by result

Thank you @richgalloway for your time