Why splunk is not comparing all values from 2 different fileds?

[chandana204]

Hi,

I want to compare two fields in a certain timerange. I am working on 2 fields, those are process_ip and transfer_ip. These process and transfer fields has 100,000 values, process_ip count is greater than transfer_ip count most of the values are same except 1 to 20 process_ip values are not matching with transfer_ip values. I wanted to list out those 1-20 process_ip values. I have written below code to compare two fields:

index="index1" process="Process" OR transfer="transfer" 
| rex "(?P[a-zA-Z0-9.]+)" 
| rex "(hd)\s+(?P[a-zA-Z0-9.]+)" 
| streamstats count by ip, ip_P 
| stats values(ip_P) AS process_ip, values(ip) AS transfer_ip 
| mvexpand process_ip 
| fillnull value=NULL transfer_ip 
| eval file_types_process=if(process_ip!=transfer_ip, process_ip, NULL) 
| where ip_process!="NULL" 
| table ip_process 

it's not effecting properly on large amount of data. When I mentioned unmatched process_ip values in the search then it's listing properly as non_matched process_ip with transfer_ip in the table ip_process. I tried running this code on 3 modes(Verbose/Smart/Fast) but no luck.

Why Splunk is not able to compare two fields accurately on large amount of data?

Please let me know if i did any mistake in the code or missed anything.

Thanks,
Chandana

[somesoni2 · Accepted solution]

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.