How to match an ip address to cidr in lookup table?

[commanman]

Hey there Splunk hero's,

Story/Background:

So, there is this variable called "src_ip" in my correlation search. The "src_ip" is a more than 5000+ ip address.

What i am doing is matching these ip address which should not be in a particular CIDR range using cidrmatch function which works prefectly.

Which looks something like this :

| where (NOT cidrmatch("34.20.223.128/25",src_ip) AND NOT cidrmatch("13.9.22.0/25",src_ip) AND NOT cidrmatch("13.56.21.18/25",src_ip) AND NOT cidrmatch("35.17.29.0/26",src_ip) AND NOT(many-more,src_ip))

SOLUTION REQUIRED:

Now, coming to the part where i need your help is .

I want to simply this.

SOLUTION Tried:
PART 1:
Solutions which i have searched over the forum tell me to create a lookup table and look through it.

So, I have created a lookup table named "match_cidr.csv". This csv/lookup file consist of more that 100+ CIDR blocks under a variable called cidr_match_src_ip.

What i have tried looking into this via this command. there is a tstat command as well so,

Query looks like this 

[ | inputlookup match_cidr.csv | where src_ip != cidr_match_src_ip] ===> this won't work since i am comparing a CIDR to IP address directly.

where NOT cidrmatch([| inputlookup match_cidr.csv], src) ==> tried this as well

What can i use here or what other things can you recommend me to do. Feel free to ask any more question to me if my message isn't clear

[yuanliu]

The problem is in your query (and possibly your lookup definition).  Generally speaking, using inputlookup as subsearch is not a solution in cases where you really mean a lookup match.

Splunk's lookup definition allows you to specify match type, one of them is CIDR.  You'll need to check "Advanced options" to add it.

Suppose your file match_cidr.csv contains the following

subnet	known_network
10.100.0.0/16	mine
10.16.5.0/24	friends
10.16.205.0/24	enemies

and the lookup name is also match_cidr.csv.  Instead of inputlookup, the correct command is lookup.

| lookup match_cidr.csv subnet AS src_ip
| where isnull(known_network)

This will find all src_ip's that are not your own, not your friends', and not your known enemies'.

[shivanshu1593]

Hello @commanman,

Problem is that when you put CIDR blocks directly in a search, it gets treated a string, not a CIDR. Hence it never matches with the IP addresses.

To resolve this, you can try the following with your lookup, that you have created:

Go to settings -> lookups -> lookup definitions and see if a definition for your lookup exists or not (It should, cos that's required to make it available for searching.). If not there, then please create one.

In the definition for your lookup, open it and click Advanced options and under Match Type, Please enter the following and click save.

CIDR(cidr_match_src_ip)

Then try the following:

| Your base search

| search NOT [| inputlookup your_lookup_name.csv | rename cidr_match_src_ip as src_ip | fields src_ip]

| Rest of your query.

Let me know if it helps.

Thanks,

S

***If this helped, please accept it as a solution. It helps others to find the solution for similar issues quickly.***

[PickleRick]

Just for the record - it is not true.

A run-anywhere example showing that search by CIDR ranges work

| makeresults
| eval ip=split("192.168.1.12,172.16.0.1,10.0.0.1",",")
| mvexpand ip
| search ip IN (192.168.0.0/16,10.0.0.0/8) 

[sjbriggs]

@PickleRick 

I recreated my lookup definition and it seems to be working now so disregard my earlier message.   Not sure what happened there but at least it seems to be doing what I want now.

[sjbriggs]

@PickleRick 

I've never had a problem with CIDR based searches in the past.  I seem to be stumbling when doing a CIDR search involving TSTATS.  I'm pulling proxy metrics based on src addresses using tstats and then attempting to limit those results to subnets listed in a lookup table and not successful at all.  The example code below works fine when my "serverlist" lookup table has individual IP addreses but returns nothing when I have CIDR addresses despite the fact that I have the lookup definition defined.  Again, i've done CIDR based lookups a ton and never had an issue but haven't done it with TSTATS before.  

| tstats sum(proxy_metrics.bytes_in) AS TotalBytesIN sum(proxy_metrics.bytes_out) AS TotalBytesOUT from datamodel=proxy_stats BY proxy_metrics.src
| search [| inputlookup serverlist
| rename subnet AS proxy_metrics.src
| fields proxy_metrics.src]
| stats sum(TotalBytesIN) AS TotalBytesIN sum(TotalBytesOUT) AS TotalBytesOUT  BY proxy_metrics.src

[commanman]

Hey @shivanshu1593 ,

As you suggested,

What I have a tried is this:

Here is my correlation search:

| tstats summariesonly=true allow_old_summaries=true values(IDS_Attacks.dest) as "dest" dc(IDS_Attacks.dest) as "count" values(IDS_Attacks.signature) as signature from datamodel="Intrusion_Detection"."IDS_Attacks" where IDS_Attacks.severity!="informational" by "IDS_Attacks.src" "IDS_Attacks.severity"

| rename "IDS_Attacks.src" as "src" "IDS_Attacks.severity" as severity

| where 'count'>25

| search (src!=10.161.35.50 AND src!=161.16.157.154 AND src!=161.16.155.105 AND src!=161.16.156.118 AND src!=165.249.145.180 AND src!=10.160.8.170 AND src!=10.160.8.171 AND src!=10.160.8.172 AND src!=10.160.8.173 AND src!=10.172.8.170 AND src!=10.172.8.171)

//Part where in want to input a lookup table
| where ( NOT cidrmatch("13.59.252.0/25",src) AND NOT cidrmatch("13.56.21.128/25",src) AND NOT cidrmatch("35.177.219.0/26",src) AND NOT cidrmatch("13.210.1.64/26",src) AND NOT cidrmatch("54.175.125.192/26",src) AND NOT cidrmatch("54.219.188.128/26",src) AND NOT cidrmatch("54.93.254.128/26",src) AND NOT cidrmatch("54.255.254.0/26",src) AND NOT cidrmatch("10.161.69.43/32",src) AND NOT cidrmatch("10.161.69.44/32",src) AND NOT cidrmatch("161.16.157.80/32",src) AND NOT cidrmatch("161.16.157.64/32",src) AND NOT cidrmatch("161.16.157.138/32",src) AND NOT cidrmatch("161.16.157.213/32",src) AND NOT cidrmatch("161.16.157.216/32",src))



So here i have created a lookup table having all the CIDR BLOCKS  in a excel with .csv extension.

STEP1:

My excel file looks like this:

Step 2:

Now I have converted this to .csv file

Step3:

Uploaded in lookup table.

Step 4:
Checking if file uploaded successfully:

Step 5:

Created a lookup definition as you suggested

Go to settings -> lookups -> lookup definitions and see if a definition for your lookup exists or not (It should, cos that's required to make it available for searching.). If not there, then please create one.

In the definition for your lookup, open it and click Advanced options and under Match Type, Please enter the following and click save.

CIDR(cidr_match_src_ip) =è mine would be CIDR(CIDR) I have reduced the complexity let me know if this wrong or not.

Step 6:
Now checking if it has been successfully created

Step 7:

Now that every thing is in place lets run this query.

| Your base search

| search NOT [| inputlookup your_lookup_name.csv | rename cidr_match_src_ip as src_ip | fields src_ip]

| Rest of your query.

==================MY QUERY ++++++++++++++++++++++++++++++++

| tstats summariesonly=true allow_old_summaries=true values(IDS_Attacks.dest) as "dest" dc(IDS_Attacks.dest) as "count" values(IDS_Attacks.signature) as signature from datamodel="Intrusion_Detection"."IDS_Attacks" where IDS_Attacks.severity!="informational" by "IDS_Attacks.src" "IDS_Attacks.severity"

| rename "IDS_Attacks.src" as "src" "IDS_Attacks.severity" as severity

| where 'count'>25

| search (src!=10.161.35.50 AND src!=161.16.157.154 AND src!=161.16.155.105 AND src!=161.16.156.118 AND src!=165.249.145.180 AND src!=10.160.8.170 AND src!=10.160.8.171 AND src!=10.160.8.172 AND src!=10.160.8.173 AND src!=10.172.8.170 AND src!=10.172.8.171)

//Part where in will insert our methodology
 | search NOT [ | inputlookup cidr.csv | rename CIDR as src | fields src ]

This does not works.

ERROR I AM FACING ARE:

The output stats is also not correct.

If you could correct me where I am wrong would be great.

Thanks,
@commonman

[sjbriggs]

Where did you end up with this?

I'm having issues as well but my errors are that my tstats search doesn't seem to like searching for src IPs that are CIDRs.  When I use a lookup table full of regular IP addresses, I get results but when I use a lookup table full of CIDR addresses I don't get any results despite having created the lookup definition and such.

[PickleRick]

Most probably nowhere, judging from lack of accepted solution and a generally dead thread.

You'll probably have a better chance of reasonable response if you create a new thread with verbose description of what you have, what you want to achieve and what you tried so far.