Splunk Search

Extract only global IP with rex

kuga_mbsd
New Member

Hi there,
I want to extract only global IP addresses of destination from the internet access logs.
Our server segments has both 10.0.0.0/8 and 192.168.0.0/16 exist and web-proxy records even from PC to 192.168.0.0/16 and 10.0.0.0/8 servers.
I could extract either 10.0.0.0/8 or 192.168.0.0/16 with rex, however cannot get not matching both these private networks.
How can I make it?

Tags (1)
0 Karma
1 Solution

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

View solution in original post

0 Karma

kuga_mbsd
New Member

OMG, it's embarrassing...
Yeah, cidrmatch definitely works in this case!

Thanks a lot, both laserval and stephanefotso!!

0 Karma

stephanefotso
Motivator

OK. Means you just want to pick www.google.com . But since www.google.com is not one of your dhost field values you must extract that value from your raw event. Here you go

index=proxy_log | rex field=_raw "http\:\/\/(?<webaccess>\w+\.\w+\.\w+)"|table webaccess

Thanks

SGF
0 Karma

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

0 Karma

stephanefotso
Motivator

Let's get your sample event, and please can you be more specific on the global Ip you want to extract?

SGF
0 Karma

kuga_mbsd
New Member

Hi stephanefotso,
our web-proxy log is like as folllows,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

2 bottom logs are both for access to our intra servers, want to only pick up the web access to external like google.com.
I tried following, but doesnt work...
index=proxy_log | rex field=dhost "(?!^10.\d+.\d+.\d+|^192.168.\d+.\d+")

0 Karma

kuga_mbsd
New Member

oops, escape has gone...

0 Karma

stephanefotso
Motivator

If i have understood, you want to extract

www.google.com
192.168.200.10
10.0.200.10

and put them in the same field. Isn't it?
Also, is dhost a field in your events? if yes, can you let us see a sample value of that field?

SGF
0 Karma

kuga_mbsd
New Member

I want to only pick up www.google.com from the sample log since 192.168./10. are our internal web servers.
yes, dhost is our field which has destination IP address of www servers:173.194.120.67, 192.168.200.10, and 10.200.200.10.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...