How to group the errors that come between 2 events?

[kranthimutyala]

We have a couple of processes that runs regularly and I want to capture the errors and groups them run wise and date wise. I tried with transactions but its not splitting run wise and gave all the errors in the same group. Please help thanks

LogDate = "01/28/2022 03:00:47.417" , LogNo = "133" , LogLevel = "INFO" , LogType = "Bot End" , LogMessage = "Logger Session Stopped; Total run time: 0:17:22.002" , TimeTaken = "0:00:00.500" , ProcessName = "FARollforward" , TaskName = "Logger" , RPAEnvironment = "PROD" , LogId = "0133010____120220128030047417" , MachineName = "xxxxx" , User = "xxxxxx"

LogDate = "01/28/2022 03:00:38.679" , LogNo = "125" , LogLevel = "ERROR" , LogType = "Process Level" , LogMessage = "EXCEPTION: CustomSubTaskError;" , TimeTaken = "0:00:00.005" , ProcessName = "FARollforward" , TaskName = "NavigateOracle" , RPAEnvironment = "PROD" , LogId = "0125010____120220128030038679" , MachineName = "xxxxx" , User = "xxxxxx"

LogDate = "01/28/2022 01:01:47.004" , LogNo = "51" , LogLevel = "ERROR" , LogType = "Process Level" , LogMessage = "EXCEPTION: Unable to perform LEFTCLICK action. , TimeTaken = "0:00:00.017" , ProcessName = "FARollforward" , TaskName = "FARollforward-NavigateOracle" , RPAEnvironment = "PROD" , LogId = "0051010____120220128010147004" , MachineName = "xxxxxxx" , User = "xxxxxx"

LogDate = "01/27/2022 23:59:20.534" , LogNo = "1" , LogLevel = "INFO" , LogType = "Bot Start" , LogMessage = "Logger Session Started" , TimeTaken = "0:00:00.000" , ProcessName = "FARollforward" , TaskName = "Logger" , RPAEnvironment = "PROD" , LogId = "0001010____120220127235920534" , MachineName = "xxxxxx" , User = "xxxxx"

ProcessName	Errors	Date
FARollForward	EXCEPTION: CustomSubTaskError EXCEPTION: Unable to perform LEFTCLICK action	01/28/2022
Cp	EXCEPTION: CustomSubTaskError EXCEPTION: Unable to perform LEFTCLICK action Exception: Failed	02/07/2022
FARollForward	EXCEPTION: CustomSubTaskError EXCEPTION: Unable to perform LEFTCLICK action	02/08/2022

[ITWhisperer]

Assuming event stream is already sorted in the manner shown in the example i.e. latest event first

| eval group=if(LogType="Bot End",1,0)
| streamstats sum(group) as group global=f by ProcessName
| stats values(LogMessage) as LogMessage first(LogDate) as Date by ProcessName group

[kranthimutyala]

Looks like its grouping well but I'm getting all the events in between, I wanted to get only the exceptions like these if they occur 

Some of them are :

EXCEPTION: Cannot find the window or application titled 'Oracle Enterprise Performance Management System Workspace, Fusion Edition - Google Chrome' that was open during recording. at Line: 50

EXCEPTION: Please enter the source string. at Line: 27

EXCEPTION: Unable to find CLIENT. Search Criteria did not match. at Line: 120

EXCEPTION: Unable to find LABEL. Search Criteria did not match. at Line: 101

EXCEPTION: Unable to find LABEL. Search Criteria did not match. at Line: 104

I tried to add LogMessage = "Exception:*" but it's giving only the latest run details and not all . Is there any way we can get only this info.Thanks for your reply.

[ITWhisperer · Accepted solution]

| eval group=if(LogType="Bot End",1,0)
| eval exception=if(match(LogMessage,"(?i)exception"),"Y",null())
| streamstats sum(group) as group global=f by ProcessName
| where exception="Y"
| stats values(LogMessage) as LogMessage first(LogDate) as Date by ProcessName group

[PickleRick]

OK, but which field or combination of fields should distinguish one run from another?

[kranthimutyala]

Hi @PickleRick process starts with LogType = "Bot Start" And ends with LogType = "Bot end" For every run. 

[PickleRick]

So it's only about time-based sequence? Are you 100% sure that two runs cannot interleave?

[kranthimutyala]

yes, they never interfere at any time. 

[ITWhisperer]

Please can you explain a bit more about your events?

Are the fields already extracted?

Does LogNo restart at 1 and go up for each process instance?

Do your events have _time = LogDate?

[kranthimutyala]

Hi @ITWhisperer 

Yes fields are already extracted 

Whenever the process triggers it starts with LogNo 1 and it can go to any number (end LogNo will be inconsistent depends on process execution and end)  

process starts with LogType = "Bot Start" And ends with LogType = "Bot end" For every run.

Yes all the events have _time but there are instances where process runs more than once so to be precise it should differentiate the errors and group them for each run for different processes 

Thanks for the reply , let me know if any additional details are required

[ITWhisperer]

Can process instances overlap e.g. can one instance of process FARollForward start before another instance of process FARollForward finishes?

[kranthimutyala]

No that never happens, the next process run would be triggered only if the current process is finished or errored out.