topic Re: How to ignore failed seq ids? in Splunk Search

How to ignore failed seq ids?

Deepz2612 — Wed, 15 Nov 2017 14:15:47 GMT

Hello Seniors,

I wanted my results to have the information of the seq ids that are received.So i use the Keyword "receive"
But i do not want my result to have seq ids that are failed so my search should first look for the keyword "recieve" and then if is "not" failed in the next event it should be displayed.
We just have the clue that the time the seq id start to recieve is same that of the time that it failed.

Please help me with this.
Thanks in advance!!

Re: How to ignore failed seq ids?

gcusello — Wed, 15 Nov 2017 14:52:42 GMT

Hi Deepz2612,
try something like this.

index=my_index receive NOT failed
| rex "seq id \"(?<Seq_Id>\d+)"
| table _time Seq_Id

Your search could be quicker with field extraction of fields that contain "receive" and "failed".
Obviously if you already extracted Seq_Id field, you don't need to use rex command.
At the same time it should ne better (I don't know if it's possible for your logs) to use "staring to receive" instead of "NOT failed".

Bye.
Giuseppe

Re: How to ignore failed seq ids?

Deepz2612 — Tue, 29 Sep 2020 16:44:15 GMT

Thanks!
When i used the above query the seq_id which are failed to receive are also included.
But my output should exclude the seq_ids that are failed.
In my above log snippet the seq_ids 890123 & 123890 should not be included in my output.

Re: How to ignore failed seq ids?

Deepz2612 — Tue, 29 Sep 2020 16:50:19 GMT

Hi,

can I get some help on this please.
the query suggested fetches me with the seq_id which are failed to receive are also included.
But my output should exclude the seq_ids that are failed.
In my above log snippet the seq_ids 890123 & 123890 should not be included in my output

Re: How to ignore failed seq ids?

gcusello — Sat, 18 Nov 2017 10:14:13 GMT

Hi Hi Deepz2612,,
it's very strange: I ingested your example data and my search extracts only the four correct events, " failed to receive" are excluded from my results.

my search

sourcetype=my_sourcetype index=my_index receive NOT failed
 | rex "seq id \"(?<Seq_Id>\d+)"
 | table _time Seq_Id _raw

my results

_time                           Seq_Id   _raw
2017-11-07 20:37:25 1234567  11/7/17 8:37:25 PM [INFO] |receive |staring to receive |seq id "1234567"
2017-11-07 20:30:27 890123   11/7/17 8:30:27 PM [INFO] |receive |staring to receive |seq id "890123"
2017-11-07 20:28:30 567890   11/7/17 8:28:30 PM [INFO] |receive |staring to receive |seq id "567890"
2017-11-07 20:20:27 123890   11/7/17 8:20:27 PM [INFO] |receive |staring to receive |seq id "123890"

Bye.
Giuseppe

Re: How to ignore failed seq ids?

niketn — Tue, 29 Sep 2020 16:50:24 GMT

@Deepz2612, based on the data sample provided please find below the run anywhere search. The commands till eval _time are just to mock data. You would need to put the commands from rex afterward to extract send and receive as type and also sequence number as seq_id. You need to perform stats to group type values by seq_id, and then add the search filter as per your need. Following search filter find only the events with single send type.

| makeresults 
| eval data="11/7/17 8:37:25 PM [INFO] |Send|staring to send|seq id \"234567\";11/7/17 8:37:25 PM [INFO] |receive |staring to receive |seq id \"1234567\";11/7/17 8:30:27 PM [INFO] |receive |staring to receive |seq id \"890123\";11/7/17 8:30:27 PM [INFO] |receive |failed:javaException;11/7/17 8:37:25 PM [INFO] |Send|staring to send|seq id \"453789\";11/7/17 8:28:30 PM [INFO] |receive |staring to receive |seq id \"567890\";11/7/17 8:20:27 PM [INFO] |receive |staring to receive |seq id \"123890\";11/7/17 8:20:27 PM [INFO] |receive |failed:javaException" 
| makemv delim=";" data 
| mvexpand data 
| rename data as _raw 
| rex "^(?<_time>\d{1,2}\/\d{1,2}\/\d{2,4}\s\d{1,2}:\d{1,2}:\d{1,2}\s(A|P)M)" 
| eval _time=strptime(_time,"%m/%d/%y %H:%M:%S") 
| rex "\|staring to (?<type>[^\|]+)[\||\s]seq id \"(?<seq_id>[^\"]+)\"" 
| eval type=trim(type) 
| stats count as eventcount earliest(_time) as _time values(type) as type by seq_id 
| search eventcount=1 AND type="send" AND type!="receive"

Once the regular expression is tested with your sample events, you should save the same as Fields knowledge object through IFX.

Re: How to ignore failed seq ids?

Deepz2612 — Sun, 19 Nov 2017 07:20:51 GMT

In the above example seq_id 123890 was originally failed but was received successfully twice after that.

I tried with calculating the duration between the events and it dint work.

Re: How to ignore failed seq ids?

niketn — Tue, 29 Sep 2020 16:50:37 GMT

@Deepz2612, any combination is possible provided you define what is the use case. Based on your original question send and receive with seq_id was supposed to be successful scenario. However above has three receive for 123890 but no send? They should be considered as three failures.

What are the conditions for seq_id? Will same seq_id be repeated when the receive fails (in other words until it is sent successfully)? Can same seq_id reappear even after the data is sent successfully (i.e. is it really unique id for transaction or more like thread id which gets reused once it is freed)?

Re: How to ignore failed seq ids?

niketn — Fri, 24 Nov 2017 04:24:40 GMT

@Deepz2612, please accept the answer if it has worked, if not please provide further details as requested.

Re: How to ignore failed seq ids?

Deepz2612 — Tue, 29 Sep 2020 16:56:54 GMT

Now I would want to ignore the sequence id 123890 which has failed @ 8 20 PM.
But my results should have the se_id 123890 which were received at 9:20 and 10:20 PM.

Re: How to ignore failed seq ids?

niketn — Sun, 26 Nov 2017 05:28:30 GMT

Should the id 123890 at 11/7/17 10:20:27 receive or send? Originally you had mentioned received and send as successful, but based on 9:20 and 10:20 data can receive and receive also be successful?

If seq id is not unique and may be reused, then possibility is you will keep running into orphaned or incorrect transaction. Is it possible to change logging to stitch receive and send with same unique transaction id?