I have splunk logs that are of 2 types, successes and failures. They contain 2 things:
"SUCCESS" "ID: <IDNumber>"
"FAILURE" "ID: <IDNumber>"
My goal is to find IDs that are identified with failures that are not also identified with a success. So for the data:
"SUCCESS" "ID: 0000", "FAILURE" "ID: 0000", "SUCCESS" "ID: 1111", "FAILURE" "ID: 2222", "SUCCESS" "ID: 3333", "FAILURE" "ID: 4444"
the result would be the IDs 2222 and 4444
My current search is:
index=sampleindex source=samplesource "SUCCESS" | rex field=_raw "ID: (?<id1>+)" | join [search index=sampleindex source=samplesource "FAILURE" | rex field=_raw "ID: (?<id2>+)"] | table id1, id2
I am able to perform each of these searches separately and output the ids, but when I combine them I cannot get the results of either id1 or id2, so I am not able to compare them
Does anyone know how I can structure my search to achieve my final goal?
I am assuming that you will have 2 separate events, one for success and one for failure, so in that case, you'll need a slightly different version of @yuanliu SPL
index=sampleindex source=samplesource (SUCCESS OR FAILURE)
| rex field=data "\"(?<status>SUCCESS)\" \"ID:\s*(?<id>\d+)"
| rex field=data "\"(?<status>FAILURE)\" \"ID:\s*(?<id>\d+)"
| stats values(status) as status by id
| where mvcount(status)=1 AND status="FAILURE"
so the stats command combines all status values for the same id and then the where clause filters out only those ids where the status is FAILURE only
You can perform a set operation on the two searches as illustrated. But a far more efficient method is to perform just one search and use mvmap.
index=sampleindex source=samplesource (SUCCESS OR FAILURE)
| rex field=data max_match=0 "\"SUCCESS\" \"ID:\s*(?<success>\d+)"
| rex field=data max_match=0 "\"FAILURE\" \"ID:\s*(?<failure>\d+)"
| eval failed = mvmap(failure, if(failure != success, failure, null()))
(In SPL, equality test of a scalar and a vector iterates over the vector.) The field failed now contains all the failure IDs that is not found in success.