Splunk Search

Extract only global IP with rex

kuga_mbsd
New Member

Hi there,
I want to extract only global IP addresses of destination from the internet access logs.
Our server segments has both 10.0.0.0/8 and 192.168.0.0/16 exist and web-proxy records even from PC to 192.168.0.0/16 and 10.0.0.0/8 servers.
I could extract either 10.0.0.0/8 or 192.168.0.0/16 with rex, however cannot get not matching both these private networks.
How can I make it?

Tags (1)
0 Karma
1 Solution

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

View solution in original post

0 Karma

kuga_mbsd
New Member

OMG, it's embarrassing...
Yeah, cidrmatch definitely works in this case!

Thanks a lot, both laserval and stephanefotso!!

0 Karma

stephanefotso
Motivator

OK. Means you just want to pick www.google.com . But since www.google.com is not one of your dhost field values you must extract that value from your raw event. Here you go

index=proxy_log | rex field=_raw "http\:\/\/(?<webaccess>\w+\.\w+\.\w+)"|table webaccess

Thanks

0 Karma

laserval
Communicator

Given your example from the comments,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

If you have the field dhost extracted, you can use the eval function cidrmatch("X",Y):

index=proxy_log | where NOT (cidrmatch("10.0.0.0/8", dhost) OR cidrmatch("192.168.0.0/16", dhost))

You might be able to remove the pipe and the where command keyword, I didn't really have data to test that on.

0 Karma

stephanefotso
Motivator

Let's get your sample event, and please can you be more specific on the global Ip you want to extract?

0 Karma

kuga_mbsd
New Member

Hi stephanefotso,
our web-proxy log is like as folllows,

[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://www.google.com/ - DIRECT/173.194.120.67 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://192.168.200.10/- DIRECT/192.168.200.10 text/html
[05/Jun/2015:14:14:17 +0900] 476 192.168.0.10 TCP_MISS/200 5306 GET http://10.0.200.10/ - DIRECT/10.200.200.10 text/html

2 bottom logs are both for access to our intra servers, want to only pick up the web access to external like google.com.
I tried following, but doesnt work...
index=proxy_log | rex field=dhost "(?!^10.\d+.\d+.\d+|^192.168.\d+.\d+")

0 Karma

kuga_mbsd
New Member

oops, escape has gone...

0 Karma

stephanefotso
Motivator

If i have understood, you want to extract

www.google.com
192.168.200.10
10.0.200.10

and put them in the same field. Isn't it?
Also, is dhost a field in your events? if yes, can you let us see a sample value of that field?

0 Karma

kuga_mbsd
New Member

I want to only pick up www.google.com from the sample log since 192.168./10. are our internal web servers.
yes, dhost is our field which has destination IP address of www servers:173.194.120.67, 192.168.200.10, and 10.200.200.10.

0 Karma
Get Updates on the Splunk Community!

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...