Extract only global IP with rex

[kuga_mbsd]

Hi there,
I want to extract only global IP addresses of destination from the internet access logs.
Our server segments has both 10.0.0.0/8 and 192.168.0.0/16 exist and web-proxy records even from PC to 192.168.0.0/16 and 10.0.0.0/8 servers.
I could extract either 10.0.0.0/8 or 192.168.0.0/16 with rex, however cannot get not matching both these private networks.
How can I make it?

[kuga_mbsd]

OMG, it's embarrassing...
Yeah, cidrmatch definitely works in this case!

Thanks a lot, both laserval and stephanefotso!!

[stephanefotso]

OK. Means you just want to pick www.google.com . But since www.google.com is not one of your dhost field values you must extract that value from your raw event. Here you go

index=proxy_log | rex field=_raw "http\:\/\/(?<webaccess>\w+\.\w+\.\w+)"|table webaccess

Thanks

[laserval · Accepted solution]

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

[stephanefotso]

Let's get your sample event, and please can you be more specific on the global Ip you want to extract?

[kuga_mbsd]

Hi stephanefotso,
our web-proxy log is like as folllows,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

2 bottom logs are both for access to our intra servers, want to only pick up the web access to external like google.com.
I tried following, but doesnt work...
index=proxy_log | rex field=dhost "(?!^10.\d+.\d+.\d+|^192.168.\d+.\d+")

[kuga_mbsd]

oops, escape has gone...

[stephanefotso]

If i have understood, you want to extract

www.google.com
192.168.200.10
10.0.200.10
and put them in the same field. Isn't it?
Also, is dhost a field in your events? if yes, can you let us see a sample value of that field?

[kuga_mbsd]

I want to only pick up www.google.com from the sample log since 192.168./10. are our internal web servers.
yes, dhost is our field which has destination IP address of www servers:173.194.120.67, 192.168.200.10, and 10.200.200.10.