Sub Search - how do I compare a search event to an index, only displaying the table of events not found in index?

[manta0101]

I have created a search that contains a field that is unique. I am using this search to populate the index. however for some reason when I try and check to see if the record is in the index it doesn't work for me. The closest I have come is this:

| localop | rest ....

```first search key field```
|eval soar_uuid= id+"_RecordedFuture"

|append [search index=rf-alerts soar_uuid|rename soar_uuid as ExistingKey]
| table soar_uuid,triggered,rule.name,title,classification,url,ExistingKey

The above returns  a list of new records with a blank ExistingKey field, and matching keys for soar_uuid  of existing records with a blank soar_uuid field. 
If I could just populate either with the other field, then I could remove all the duplicates.
I want to remove the new records that match the existing records before writing the events to the index. appendsearch instead of append doesn't seem to return the existing records. 

[ITWhisperer · Accepted solution]

Assuming there is only one event for each soar_uuid in either of the two searches, i.e. it is unique in its search but possibly duplicated in the other search, you could do comething like this

| localop | rest ....

```first search key field```
|eval soar_uuid= id+"_RecordedFuture"

|append [search index=rf-alerts soar_uuid]
| eventstats count by soar_uuid
| table soar_uuid,triggered,rule.name,title,classification,url,count

count would then be 2 if it is duplicated in the appended search

[manta0101]

That worked, here is the updated SPL using your concept.

|eval soar_uuid= id+"_RecordedFuture"

|append [search index=rf-alerts soar_uuid]
| eventstats count by soar_uuid |where count<2
| table soar_uuid,triggered,rule.name,title,classification,url,count