Splunk Search

Finding an Error in the middle of a Transaction

tyronetv
Communicator

I have a unique ID (RID) for a each client click. A single click can execute between 3 and, sometimes, over 100 log entries. Sometimes, in the middle of all of that is a situation where an error occurs.

I currently have a search that generates a list of actions by userid/RID but the report creates every line as an "INFO" response (I guess because the first and last line of the transaction are always INFO log level). What I need to do is set the corresponding report to acknowledge the transaction as an ERROR level based upon the ERROR existing someone in the middle of the transaction.

How would I do that?

2014-02-04 05:58:14,473 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:58:14,462 ERROR [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,776 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

The above is for a client search request. The request timed out after 5 minutes. When I report on this transaction as a summary of actions taken by the client I currently report on the action and the duration. I would like to include the "Error_Level" as a column with a drill down to the actual transaction(RID) experiencing the error. But first, I just want to be able to identify it as an 'error' transaction.

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

View solution in original post

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

kristian_kolb
Ultra Champion

Maybe not exactly what you're asking for, but a general optimization tip, especially if there are a large amount of transactions, and only a few of them has errors.

Instead of creating transactions and then looking for possible ERRORs, it might be better to do it the other way round:

index=blah sourcetype=bleh [index=blah sourcetype=bleh ERROR | dedup RID | fields + RID] | transaction RID

The subsearch will find the RIDs where there are ERRORs, and the outer search will build transactions based on them.

0 Karma

tyronetv
Communicator

Yeah. I understand this as a solution but more often then not (and shame on me training them as such) the users are used to seeing a sequential listing of the user's activity and I've been asked to mark the one that Error'd as part of the report. So, I find the user, then their RID's, and then generate a report. I need the report to show the ERROR on the line item of the users activity.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...

Customer success is front and center at .conf25

Hi Splunkers, If you are not able to be at .conf25 in person, you can still learn about all the latest news ...