find related events

[indeed_2000]

Hi

How can I find event that have send but not recieved response

here is the log:

this is send
2021-07-15 00:00:01,892 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]

this is recieve
2021-07-15 00:00:11,719 INFO CUST.InEP-Server2-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]

step1: find send id 1234567

step2: find response id  9876543 due to send id 1234567, where A=A+5 AND B=B

finally show id than not have recieve

e.g

2021-07-15 00:00:01,988 INFO CUST.InAB-ServerApp-0000001 [ApiManager] Send Packet [0000000000000*] to [APP.MODULE]

table   id status

id                        status

0000001    no recieve

any idea?

thanks

[bowesmana]

Here is a simplistic example that uses your data and some other made up data as an example. Paste this into a search and have a look

| makeresults
| eval _raw="2021-07-15 00:00:01,892 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]
2021-07-15 00:00:01,988 INFO CUST.InAB-ServerApp-0000001 [ApiManager] Send Packet [0000000000000*] to [APP.MODULE]
2021-07-15 00:00:03,123 INFO CUST.InAB-ServerApp-2345678 [MyService] Packet Processed: A[51] B[0000212]
2021-07-15 00:00:11,719 INFO CUST.InEP-Server2-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]
2021-07-15 00:00:17,666 INFO CUST.InEP-Server2-8765432_CUST.InAB-ServerApp-2345678 [MyService] Normal Packet Received: A[56] B[0000212]"
| multikv noheader=t
| fields _raw
| eval _time=strptime(_raw, "%F %T,%Q")
| rex field=_raw "CUST.InAB-ServerApp-(?<SendId>\d+)[^[]*\[[^[]*\[(?<A>\d+)[^[]*\[(?<B>[^]]*)"
| stats list(*) as * list(_time) as _time by SendId B
| where mvcount(A)=1
| eval Status="Response not received"

 It makes the following assumptions based on the info you gave

• Rex statement does not require A or B in data to extract those numbers
• A will be extracted only as numbers
• B is extracted as only the data between []
• The calculation makes no comparison to check whether A=A+5 in the response

It will group all fields with common values of SendId and B and then just check if there is only a single value for A.

However, is it necessary to check that A=A+5 and is it possible that there can be other messages where A can be different and which are related to the transaction?

If so, then there is a bit more logic necessary

[indeed_2000]

Thank you for answer.

as i metion I need to check A and B because there many other events that have same structure but if I check these items able to find correct one that related to this event.

step2: find response id  9876543 due to send id 1234567, where A=A+5 AND B=B

[bowesmana]

Can you provide an richer example of data where there are events that should NOT form part of the result and different examples of A and B

[indeed_2000]

this is send
2021-07-15 00:00:01,892 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]

this is send
2021-07-15 00:00:01,892 INFO ABCD.DaQW-ParityGQQ-1231234 [MyService] Packet Processed: A[60] B[0000465]

this is send

2021-07-15 00:00:01,892 INFO MNBV.ZaQW-ChatCXZ-1478523 [MyService] Packet Processed: A[70] B[0000369]

this is recieve
2021-07-15 00:00:11,719 INFO CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]

this is recieve
2021-07-15 00:00:11,719 INFO EFGH.GaXZ-Carry2-3456789_ABCD.DaQW-ParityGQQ-1231234 [MyService] Normal Packet Received: A[65] B[0000456]

step1: find send id  1234567  , 1231234  ,  1478523    

step2: find response id  9876543  ,   3456789 due to send id 1234567  , 1231234  ,  1478523    , where A=A+5 AND B=B

finally show id than not have recieve

expected Output:

id                        status

1478523     no recieve

FYI: first part of related events work like this:

Send: CUST.InAB-ServerApp-1234567

Recieve: CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567

[bowesmana]

@indeed_2000 

Run this search - it would appear your example shows B=465 and B=456 in what you suggest is a match. I assume you have a typo...

| makeresults
| eval _raw="2021-07-15 00:00:01,892 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]
2021-07-15 00:00:01,988 INFO CUST.InAB-ServerApp-0000001 [ApiManager] Send Packet [0000000000000*] to [APP.MODULE]
2021-07-15 00:00:03,123 INFO CUST.InAB-ServerApp-2345678 [MyService] Packet Processed: A[51] B[0000212]
2021-07-15 00:00:17,666 INFO CUST.InEP-Server2-8765432_CUST.InAB-ServerApp-2345678 [MyService] Normal Packet Received: A[56] B[0000212]
2021-07-15 00:00:01,892 INFO ABCD.DaQW-ParityGQQ-1231234 [MyService] Packet Processed: A[60] B[0000465]
2021-07-15 00:00:01,892 INFO MNBV.ZaQW-ChatCXZ-1478523 [MyService] Packet Processed: A[70] B[0000369]
2021-07-15 00:00:11,719 INFO CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]
2021-07-15 00:00:11,719 INFO EFGH.GaXZ-Carry2-3456789_ABCD.DaQW-ParityGQQ-1231234 [MyService] Normal Packet Received: A[65] B[0000456]"
| multikv noheader=t
| fields _raw
| eval _time=strptime(_raw, "%F %T,%Q")
| rex field=_raw "INFO ((?<ReceiveApp>\w+\.\w+-\w+)-(?<ReceiveId>\d+)_)?(?<SendApp>\w+\.\w+-\w+)-(?<SendId>\d+)[^[]*\[[^[]*\[(?<A>\d+)[^[]*\[(?<B>[^]]*)"
| stats list(*) as * list(_time) as _time by SendApp SendId B
| eval A_Range=max(A)-min(A)

This has all the building blocks for you to achieve your search, but without knowing what all the other data is that should be ignored, it's difficult to complete it.

All you need is the SPL above from the rex statement.

Please try this against your data and give some feedback on what does not work.

[indeed_2000]

is it possible to add duration between them?

e.g.

SEND

2021-07-15 00:00:01,892 INFO CUST.InAB-ServerApp-1234567 [MyService] Packet Processed: A[50] B[0000211]

RECIEVE

2021-07-15 00:00:11,719 INFO CUST.VqPO-Oracle7-9876543_CUST.InAB-ServerApp-1234567 [MyService] Normal Packet Received: A[55] B[0000211]

output:

duration

9.827

[bowesmana]

Add this line to my previous search

| streamstats global=f range(_time) as duration by SendApp SendId B