How to search a lookup file of IP ranges without changing the format into CIDR?

vj1226 — Fri, 20 Jan 2017 14:39:53 GMT

Hello,
I have several lookup files in txt and it's in form like "blacksite1:123.123.123.1-123.123.123.17blacksite2:456.456.456.7-456.456.456.12blacksite3...."
Is there any method to use this file in the Splunk Search and Reporting app without changing the format into CIDR?
I've tried with search src_ip=123.123.123.1-123.123.123.17, obviously it doesn't work. It works well if I try to convert the range format into CIDR or regular expression, however it is a long blacklist, and some ranges should be broke down into several CIDR expressions.

Thank you in advance,

Re: How to search a lookup file of IP ranges without changing the format into CIDR?

DalJeanis — Fri, 20 Jan 2017 19:15:40 GMT

If you are worried about the size of the lookup table, you could do a strategy of breaking the IP address up into its four components, and putting the values for the blocked 4th node into a single multi value variable.

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3 
   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]
| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

The above sample data generator create block records for these IP ranges

blacksite1:123.123.123.1-123.123.123.17
blacksite2:123.123.123.20-123.123.123.24
blacksite3:456.456.456.7-456.456.456.12

using this code

   [| makeresults count=17 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 
    | append  
      [| makeresults count=5 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4 | eval IP4 = IP4+19]
    | append  
      [| makeresults count=6 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4 | eval IP4 = IP4+6]
    | eval IP4_3 = case(len(IP4)=3,IP4,len(IP4)=2,0.IP4,00.IP4)
    | stats values(IP4_3) as IP4List_3 by IP1 IP2 IP3 
    ]

and then checks to see if the lookup (here coded as a left join) gets the right results for the first 25 IP4s in 123.123.123, 456.456.456, and 789.789.789, generated by this code

| makeresults count=25 | eval IP1=123, IP2=123, IP3=123 | streamstats count as IP4
| append 
   [| makeresults count=25 | eval IP1=456, IP2=456, IP3=456 | streamstats count as IP4]
| append 
   [| makeresults count=25 | eval IP1=789, IP2=789, IP3=789 | streamstats count as IP4]
| table IP1 IP2 IP3 IP4
| eval FullIP=IP1.".".IP2.".".IP3.".".IP4
| eval IP4_3 = case(len(IP4)==3,IP4 ,len(IP4)==2,"0".IP4 ,true(),"00".IP4)

| join type=left  IP1 IP2 IP3  [the lookup table produced above]

then applies this test to see what returned, and pretties up the result with this -

| eval IP4Check=if(mvfind(IP4List_3,IP4_3)==0,"Blocked","NotBlocked")

| sort 0 FullIP
| table FullIP IP1 IP2 IP3 IP4 IP4Check IP4_3 IP4List_3

NOTE - code edited to format and use a 3-digit IP4, since a blocked IP4 =1 resulted in blocking all IP addresses with a 1 in them, and so on. OOPS.

Also edited to use sort 0 instead of sort in case there were more than 100 values to be sorted.

Re: How to search a lookup file of IP ranges without changing the format into CIDR?

vj1226 — Thu, 30 Mar 2017 10:13:22 GMT

It's complicated but works well. Thank you!

Re: How to search a lookup file of IP ranges without changing the format into CIDR?

DalJeanis — Thu, 30 Mar 2017 18:52:23 GMT

Glad to help!

topic How to search a lookup file of IP ranges without changing the format into CIDR? in Splunk Search

How to search a lookup file of IP ranges without changing the format into CIDR?

Re: How to search a lookup file of IP ranges without changing the format into CIDR?

Re: How to search a lookup file of IP ranges without changing the format into CIDR?

Re: How to search a lookup file of IP ranges without changing the format into CIDR?