Search events - remove events matching certain values on 3 fields

[Dirkoh]

Currently search will display events with "Rejected" File Status, but if this Rejected file gets fixed and then is "Delivered", I want to remove the previous "Rejected" event from the report.

How would Splunk be able to accomplish this?

In other words :

Only want Splunk to display results when field "Failures_Reason_RegEx" matches "Rejected", but want to remove all "Rejected" status' that have been fixed and resent, and are now "Delivered" status.

Do not want to actually display any "Delivered" status on the report.

I am using the Consumed_FileName_REGEX, SenderRoutingID_RegEX, and ReceiverRoutingId_RegEX in combination with each other as unique identifiers to match each event.

See below for base search which shows all "Rejected" & "Delivered" messages - I only want it to show Rejected and remove any Rejected that were Delivered at a later time.

index=CUSTOM-INDEX AND sourcetype="events" AND (host="server1" OR host="server2") AND ("Messaging.Message.MessageRejected" OR "Messaging.Message.PayloadDelivered")
| rex field=_raw "\sSenderRoutingId\((?P<SenderRoutingID_RegEX>.*)\)\sReceiverRoutingId"
| rex field=_raw "\sReceiverRoutingId\((?P<ReceiverRoutingId_RegEX>.*)\)\sDirection\("
| rex field=_raw "\sMessageState\((?P<File_Status_REGEX>.*)\)\sFinalState\("
| rex field=_raw "\sConsumptionFilename\((?P<Consumed_FileName_REGEX>.*)\)\sProductionFilename\("
| rex field=_raw "PeerAddress\((?P<Delivery_URL_RegEx>.*)\)\sConsumptionFilename\("
| rex field=_raw "Exchange\((?P<Exchange_Name_RegEx>.*)\)\sTransport\("
| rex field=_raw "RejectedReason\((?P<Failures_Reason_RegEx>.*)\)\sCycleId\("
| top limit=50000 Exchange_Name_RegEx, SenderRoutingID_RegEX, File_Status_REGEX, Consumed_FileName_REGEX, ReceiverRoutingId_RegEX, Delivery_URL_RegEx

[richgalloway]

Assuming "Exchange" is a unique identifier then one can group events by Exchange and then display only those with a single "Rejected" status.

index=CUSTOM-INDEX AND sourcetype="events" AND (host="server1" OR host="server2") AND ("Messaging.Message.MessageRejected" OR "Messaging.Message.PayloadDelivered")
| rex field=_raw "\sSenderRoutingId\((?P<SenderRoutingID_RegEX>.*)\)\sReceiverRoutingId"
| rex field=_raw "\sReceiverRoutingId\((?P<ReceiverRoutingId_RegEX>.*)\)\sDirection\("
| rex field=_raw "\sMessageState\((?P<File_Status_REGEX>.*)\)\sFinalState\("
| rex field=_raw "\sConsumptionFilename\((?P<Consumed_FileName_REGEX>.*)\)\sProductionFilename\("
| rex field=_raw "PeerAddress\((?P<Delivery_URL_RegEx>.*)\)\sConsumptionFilename\("
| rex field=_raw "Exchange\((?P<Exchange_Name_RegEx>.*)\)\sTransport\("
| rex field=_raw "RejectedReason\((?P<Failures_Reason_RegEx>.*)\)\sCycleId\("
| stats values(*) as * by Exchange_Name_RegEx
| where (mvcount(Failures_Reason_RegEx)=1 AND mvindex(Failures_Reason_RegEx,0)="Rejected")
| table Exchange_Name_RegEx, SenderRoutingID_RegEX, File_Status_REGEX, Consumed_FileName_REGEX, ReceiverRoutingId_RegEX, Delivery_URL_RegEx

[Dirkoh]

After adding your suggestion, I'm not seeing some "Rejected" messages that should still display on the report.

For example, there is a failed file in Rejected state and there is no Delivered status for this filename after the Rejected for Exchange_1 but it still does not show up on this report. 

Any ideas?

[richgalloway]

My answer presumes there are only two status values: Rejected and Delivered.  If there are more than one then the query should be modified.

index=CUSTOM-INDEX AND sourcetype="events" AND (host="server1" OR host="server2") AND ("Messaging.Message.MessageRejected" OR "Messaging.Message.PayloadDelivered")
| rex field=_raw "\sSenderRoutingId\((?P<SenderRoutingID_RegEX>.*)\)\sReceiverRoutingId"
| rex field=_raw "\sReceiverRoutingId\((?P<ReceiverRoutingId_RegEX>.*)\)\sDirection\("
| rex field=_raw "\sMessageState\((?P<File_Status_REGEX>.*)\)\sFinalState\("
| rex field=_raw "\sConsumptionFilename\((?P<Consumed_FileName_REGEX>.*)\)\sProductionFilename\("
| rex field=_raw "PeerAddress\((?P<Delivery_URL_RegEx>.*)\)\sConsumptionFilename\("
| rex field=_raw "Exchange\((?P<Exchange_Name_RegEx>.*)\)\sTransport\("
| rex field=_raw "RejectedReason\((?P<Failures_Reason_RegEx>.*)\)\sCycleId\("
| stats values(*) as * by Exchange_Name_RegEx
| where (mvcount(Failures_Reason_RegEx)=1 AND mvindex(Failures_Reason_RegEx,0)="Rejected" AND mvindex(Failures_Reason_RegEx,1)!="Delivered")
| table Exchange_Name_RegEx, SenderRoutingID_RegEX, File_Status_REGEX, Consumed_FileName_REGEX, ReceiverRoutingId_RegEX, Delivery_URL_RegEx