topic Re: How to find correct logs to find error count. in Splunk Search

How to find correct logs to find error count.

avi7326 — Mon, 04 Sep 2023 15:33:01 GMT

I want to calculate the error count from the logs . But the error are of two times which can be distinguish only from the flow end event. i.e [ flow ended put :sync\C2V]

So what condition I can put so that I can get this information from the above given log.

index=us_whcrm source=MuleUSAppLogs sourcetype= "bmw-crm-wh-xl-retail-amer-prd-api" ((severity=ERROR "Transatcion") OR (severity=INFO "Received Payload"))

I am using this query to get below logs. Now I want a condition that when it is severity=error then I can get the severity= info event of received payload to get the details of the correlationId and also end flow event so that I can determine the error type.

Re: How to find correct logs to find error count.

richgalloway — Mon, 04 Sep 2023 18:26:32 GMT

The sample INFO event does not contain a "Received Payload" text.

What field(s) link the ERROR event to an INFO event?

Re: How to find correct logs to find error count.

bowesmana — Mon, 04 Sep 2023 22:58:53 GMT

It looks like you posted the same image twice, but I am assuming that in the INFO message was the first one at 11:15:54:355 and the error was 1 millisecond earlier at 11:15:54:354 and you want to extract the ID 0021d100-46c2-11ee-9327-12b7e80d647b and then count those IDs which have only INFO and those that have both and error. Or it might be that you just want to count errors vs info

so you could do

| eval isInfo=if(severity="INFO", 1, 0) | eval isError=if(severity="ERROR", 1, 0) | stats sum(isInfo) as Transactions sum(isError) as Errors

which would just count the INFO and ERROR events, or you could do this

| rex field=message "(?<tx_id>\w{8}-\w{4}-\w{4}-\w{4}-\w{12})" | stats count by tx_id | where count=2

which would give you all the transactions that ended in error, but it depends exactly what your output requirement and also whether you have more than one possible INFO/ERROR event in the dataset.

Re: How to find correct logs to find error count.

avi7326 — Tue, 05 Sep 2023 12:27:20 GMT

This is the received payload.

index=us_whcrm source=MuleUSAppLogs sourcetype= "bmw-crm-wh-xl-retail-amer-prd-api" ((severity=ERROR "Transatcion") OR (severity=INFO "Received Payload")) | rex field=message "(?<json_ext>\{[\w\W]*\})" | table _time properties.correlationId json_ext | spath input=json_ext | rename properties.correlationId as correlationId processRetailDeliveryReporting.processRetailDeliveryReportingDataArea.retailDeliveryReporting.retailDeliveryReportingVehicleLineItem.vehicle.vehicleID as VinId | eval BMWUnit=replace(BMWUnit,"([file://w%7b3%7d)(/w%7b2%7d]\\w{3})(\\w{2})", \\1-\\2) | table _time correlationId BMWUnit dealerId Description VinId | stats earliest(_time) as _time values(*) as * by correlationId | where isnotnull(Description)

I am using this query to get all the errors and their field details in the table and it is working but now there is one condition that I have to differentiate that error they are of two types one we can get from the flow end event [sync/c2v] which I shared. And these errors I am calculating from description field.
what could I do chnage in my query to find the- error type.

Re: How to find correct logs to find error count.

avi7326 — Tue, 05 Sep 2023 09:42:10 GMT

I want a condition like when it is severity=ERROR then show its received payload event and if it has sync/C2V event then it is COO error and if it does not have that then it is RDR error.Is there any way please help me in this.
Thanks

Re: How to find correct logs to find error count.

yuanliu — Wed, 06 Sep 2023 07:35:16 GMT

First, about data illustration.

Always use text illustration; when illustrating structure data such as JSON, always use raw text. Screenshot is extremely difficult for volunteers to comprehend; they oftentimes also distort characteristics.
Always anonymize illustrated data. (One of your screenshots seems to contain information you do not want to divulge.)
Illustrate how you envision the output, use mockup.
Explain logic between illustrated data and desired output if it is not obvious. (It may be obvious to you but not to volunteers who are not intimately familiar with your data and use case.)

This said, your later code suggests that

key information "sync/c2v" is contained in the non-JSON part of "message". You need to extract that first (I'll put it in leadtext), then evaluate Description based on whether that information is present.
there is a field properties.correlationId that links ERROR event and INFO events, therefore transaction ID that is also contained in the non-JSON part of message is not needed. Is this correct?

If these guesses are correct, you are looking for something like

index=us_whcrm source=MuleUSAppLogs sourcetype= "bmw-crm-wh-xl-retail-amer-prd-api" ((severity=ERROR "Transatcion") OR (severity=INFO "Received Payload")) | rex field=message "(?<leadtext>[^{]+):\s*(?<payload>{.+})" | eval Description = if(match(leadtext, "flow ended put:\\sync\\c2v"), "COO", "RDR") | spath input=payload | rename properties.correlationId as correlationId processRetailDeliveryReporting.processRetailDeliveryReportingDataArea.retailDeliveryReporting.retailDeliveryReportingVehicleLineItem.vehicle.vehicleID as VinId | eval BMWUnit=replace(BMWUnit,"([file://w%7b3%7d)(/w%7b2%7d]\\w{3})(\\w{2})", "\\1-\\2") | table _time correlationId BMWUnit dealerId Description VinId | stats earliest(_time) as _time values(*) as * by correlationId | where Description == "COO"

Hope this helps