Splunk Search

Matching a lookup table using rex or regex to match to data in the result of a query field

CyberCyberSec
Loves-to-Learn

Hello,

 

I am trying to use a lookup table to search against the URL field inside of the proxy logs. The use case is to find out if any users have been accessing any domains/URL's that is listed in the lookup file. I am trying to strip away all the extra characters and keep the url/domain that it is matching on.

This is what I have so far:

 

index=proxy sourcetype=proxy:syslog:proxy_web_policy[|inputlookup lookupfile_last_status  |return 1000 $lookup_domain_name]|rex field=proxy_url_field "(?<lookup_domain_name>(\w+\.)+\w+)"|table name url status

Example:

www.google.com/complete/search?client=chrome-omni&gs_ri=chrome-ext-ansg&xssi=t&q=gist.github.com/mcaj-admin/18558a1ec6a782d2452f971e806230c6&oit=3&url=https://gist.github.com/mcaj-admin/18558a1ec6a782d2452f971e806230c6&pgcl=4&gs_rn=42&psi=VqC6Zso_KQ3pgOGL&sugkey=AIzaSyBOti4mM-6x9WDnZIjIeyEU21OpBXqWBgw  

is is matching on "gist.github.com/mcaj-admin/18558a1ec6a782d2452f971e806230c6"

How can I strip away everything but what it is matching on? I am trying to figure out how to use regex + the lookup

 

Thanks.

 

Labels (1)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

remove [#This will extract domain from url] from below search.

index=proxy sourcetype=proxy:syslog:proxy_web_policy 
| stats count as event_count by url
| rex field=url "^(\w+:\/\/)?(?<domain>[^\/]+)" #This will extract domain from url
| append [|inputlookup lookupfile_last_status | stats count as lookup_count by lookup_domain_name  | rename lookup_domain_name as domain  ]
| stats values(*) as * by domain
| where isnotnull(event_count) AND isnotnull(lookup_count)

 

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2025 SplunkTrust is officially open! If you ...

Splunk Answers Content Calendar, June Edition II

Get ready to dive into Splunk Dashboard panels this week! We'll be tackling common questions around ...

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...