topic Re: How to search across all my data for any public IP addresses? in Splunk Search

How to search across all my data for any public IP addresses?

j4adam — Wed, 08 Jun 2016 14:19:07 GMT

Hello all,

I've been hoping to play around with some of the iplocation functionality and see if I could leverage it somehow, so I rooted around and found that most of my data is private addresses. That was to be expected, but as I dig around, I cannot seem to find any public addresses at all.

From there I thought it would be fairly simple to do a search across all my data for any public address, or at the very least any non-private (weed out the smaller set). It turns out that I cannot find an easy way of doing this. Regex sounds like it would be a good approach, but that alone as proven to be more complex than expected.

Since I'm looking to find a public IP in any location, which means I'm not specifying a source, sourcetype, or field. So I'm either using _raw with regex or index=* searches for IP addresses. Without regex, it becomes a bit of a bear because doing a NOT search without specifying a field (which I do not know) removes the whole event, which may also contain a public address.

I've found a couple regex online that match RFC 1918 addresses, but most use the /m flag in regex101 (m modifier: multi-line. Causes ^ and $ to match the begin/end of each line) which it looks like splunk does not use.

I'm currently using this:

index=*| regex _raw="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}" | search src_ip!=10.* src_ip!=127.* src_ip!=172.16.* src_ip!=::ffff:10* src_ip.!=::ffff:127.* src_ip!=::fff:172.16.*

Ideally I'd not have to specify a field, as I said previously, but my current approach is to pair down the data with specific qualifiers until I either find what I'm looking for or run out of data.

I very well may not have any public addresses in the data I'm using!

Any suggestions for this problem?

Re: How to search across all my data for any public IP addresses?

pgreer_splunk — Mon, 27 Jun 2016 14:57:54 GMT

Maybe the regex's below might help in being more specific to just public ranges?

"172.(1[6-9].|2[0-9].|3[0-1].)[0-9]{1,3}.[0-9]{1,3}"

"(10.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})"

Maybe that might find some matches for your initial regex against _raw events.

Re: How to search across all my data for any public IP addresses?

dkadavis — Fri, 18 Oct 2019 14:22:48 GMT

Just adding to pgreer's answer

| makeresults 
| eval ip_list="172.16.20.1,10.1.1.1,192.168.1.1,1.2.3.4,127.0.0.1,169.254.20.10"
| makemv ip_list delim="," 
| mvexpand ip_list
| eval ip_type = case(match('ip_list',"172.(1[6-9].|2[0-9].|3[0-1].)[0-9]{1,3}.[0-9]{1,3}"),"1_private",match('ip_list',"(10.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})"),"1_private",match('ip_list',"(192.168.[0-9]{1,3}.[0-9]{1,3})"),"1_private",match('ip_list',"(127.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})"),"3_loopback",match('ip_list',"(169.254.[0-9]{1,3}.[0-9]{1,3})"),"2_apipa",1=1,"0_public")
| sort ip_type ip_list

Re: How to search across all my data for any public IP addresses?

fatsug — Fri, 02 Sep 2022 10:17:11 GMT

This looks really nice!

How would I populate "ip_list" with output from my own "base search"? Or do some direct pipe to eval?