Splunk Search

Why splunk is not comparing all values from 2 different fileds?

chandana204
Communicator

Hi,

I want to compare two fields in a certain timerange. I am working on 2 fields, those are process_ip and transfer_ip. These process and transfer fields has 100,000 values, process_ip count is greater than transfer_ip count most of the values are same except 1 to 20 process_ip values are not matching with transfer_ip values. I wanted to list out those 1-20 process_ip values. I have written below code to compare two fields:

index="index1" process="Process" OR transfer="transfer" 
| rex "(?P[a-zA-Z0-9.]+)" 
| rex "(hd)\s+(?P[a-zA-Z0-9.]+)" 
| streamstats count by ip, ip_P 
| stats values(ip_P) AS process_ip, values(ip) AS transfer_ip 
| mvexpand process_ip 
| fillnull value=NULL transfer_ip 
| eval file_types_process=if(process_ip!=transfer_ip, process_ip, NULL) 
| where ip_process!="NULL" 
| table ip_process 

it's not effecting properly on large amount of data. When I mentioned unmatched process_ip values in the search then it's listing properly as non_matched process_ip with transfer_ip in the table ip_process. I tried running this code on 3 modes(Verbose/Smart/Fast) but no luck.

Why Splunk is not able to compare two fields accurately on large amount of data?

Please let me know if i did any mistake in the code or missed anything.

Thanks,
Chandana

0 Karma
1 Solution

somesoni2
Revered Legend

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

View solution in original post

somesoni2
Revered Legend

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...