How to identify external IP addresses

[brandonmurphy]

I am attempting to identify external IPs that are accessing our servers more than a given number of times each day in order to identify external users that are pulling data from our site using automated processes.

The plan is to identify the IPs, then associate those IPs with various services that are using the data in order to create points of contact. If anyone could help, I would really appreciate it.

[woodcock]

You can do something like this:

index="YouShouldAlwaysSpecifyAnIndex" AND sourcetype="AndSourcetypeToo"
| rename COMMENT AS "We are assuming that the 'src_ip' has the source IP address value and that 'MyCompany.com' is your domain."
| lookup dnslookup clientip AS src_ip OUTPUT clienthost
| eval internal = if(match(clienthost, "(?i)*.MyCompany.com$") OR  match(src_ip, "(192\.168\.)\d{1,3}\.\d{1,3}|(10\.)\d{1,3}\.\d{1,3}\.\d{1,3}|(172\.1[6-9]\.)\d{1,3}\.\d{1,3}|(172\.2[0-9]\.)\d{1,3}\.\d{1,3}|(172\.3[0-1]\.)\d{1,3}\.\d{1,3}"), 1, 0)
| where internal == "1"
| stats count dc(dest) BY src_ip
| where count >= YourThresholdNumberHere

[ololdach]

Hi Brandon, with the limited information at hand this is how I would do it:
... search that gives you the ip as **dst** field | lookup dnslookup clientip as dst OUTPUT clienthost as dst_resolved
Try it with:

| makeresults | eval dst="8.8.8.8" | lookup dnslookup clientip as dst OUTPUT clienthost as dst_resolved

Now you have the host resolution with the domain name. Filter the results by adding | where NOT dst_resolved like "%.yourdomain"This will give you all events with an external domain.

If you have multiple internal domains that change on a daily basis, I suggest another method along name resolution: External DNS servers like 8.8.8.8 will never resolve your internal IPs. Write a small python custom search command, that will try to resolve the IP externally. If it fails, it is an internal address. There are some code snippets around, if you want to go that way.

[wmyersas]

Do you have a CIDR for internal addresses to ignore?

[brandonmurphy]

Not that I am aware of

[wmyersas]

If you can't identify internal-vs-external addresses, it's going to be hard to help you out 🙂

[aohls]

Do you have an example of the logs?

[brandonmurphy]

Not exactly. I can't post from the system it is on. I was hoping for something fairly general that I can work with to meet my needs.