Splunk Search

Why splunk is not comparing all values from 2 different fileds?

chandana204
Communicator

Hi,

I want to compare two fields in a certain timerange. I am working on 2 fields, those are process_ip and transfer_ip. These process and transfer fields has 100,000 values, process_ip count is greater than transfer_ip count most of the values are same except 1 to 20 process_ip values are not matching with transfer_ip values. I wanted to list out those 1-20 process_ip values. I have written below code to compare two fields:

index="index1" process="Process" OR transfer="transfer" 
| rex "(?P[a-zA-Z0-9.]+)" 
| rex "(hd)\s+(?P[a-zA-Z0-9.]+)" 
| streamstats count by ip, ip_P 
| stats values(ip_P) AS process_ip, values(ip) AS transfer_ip 
| mvexpand process_ip 
| fillnull value=NULL transfer_ip 
| eval file_types_process=if(process_ip!=transfer_ip, process_ip, NULL) 
| where ip_process!="NULL" 
| table ip_process 

it's not effecting properly on large amount of data. When I mentioned unmatched process_ip values in the search then it's listing properly as non_matched process_ip with transfer_ip in the table ip_process. I tried running this code on 3 modes(Verbose/Smart/Fast) but no luck.

Why Splunk is not able to compare two fields accurately on large amount of data?

Please let me know if i did any mistake in the code or missed anything.

Thanks,
Chandana

0 Karma
1 Solution

somesoni2
Revered Legend

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

View solution in original post

somesoni2
Revered Legend

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...