Splunk Search

Finding an Error in the middle of a Transaction

tyronetv
Communicator

I have a unique ID (RID) for a each client click. A single click can execute between 3 and, sometimes, over 100 log entries. Sometimes, in the middle of all of that is a situation where an error occurs.

I currently have a search that generates a list of actions by userid/RID but the report creates every line as an "INFO" response (I guess because the first and last line of the transaction are always INFO log level). What I need to do is set the corresponding report to acknowledge the transaction as an ERROR level based upon the ERROR existing someone in the middle of the transaction.

How would I do that?

2014-02-04 05:58:14,473 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:58:14,462 ERROR [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,776 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

The above is for a client search request. The request timed out after 5 minutes. When I report on this transaction as a summary of actions taken by the client I currently report on the action and the duration. I would like to include the "Error_Level" as a column with a drill down to the actual transaction(RID) experiencing the error. But first, I just want to be able to identify it as an 'error' transaction.

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

View solution in original post

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

kristian_kolb
Ultra Champion

Maybe not exactly what you're asking for, but a general optimization tip, especially if there are a large amount of transactions, and only a few of them has errors.

Instead of creating transactions and then looking for possible ERRORs, it might be better to do it the other way round:

index=blah sourcetype=bleh [index=blah sourcetype=bleh ERROR | dedup RID | fields + RID] | transaction RID

The subsearch will find the RIDs where there are ERRORs, and the outer search will build transactions based on them.

0 Karma

tyronetv
Communicator

Yeah. I understand this as a solution but more often then not (and shame on me training them as such) the users are used to seeing a sequential listing of the user's activity and I've been asked to mark the one that Error'd as part of the report. So, I find the user, then their RID's, and then generate a report. I need the report to show the ERROR on the line item of the users activity.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...