Splunk Enterprise Security

Multiple Domains

CyberWolf
Path Finder

I have an SPLQ that im trying to collect all domains from a raw logs, but my regex is capturing only one domain.
in a single event, some events have one url some of them have 20 and more, how do i capture all domains, please advice?



SPLQ
..............
| rex field=_raw "(?<domain>\w+\.\w+)\/"
| rex field=MessageURLs "\b(?<domain2>(?:http?://|www\.)(?:[0-9a-z-]+\.)+[a-z]{2,63})/?"
| fillnull value=n/a
| stats count by domain domain2 MessageURLs _raw

Labels (1)
Tags (1)
0 Karma
1 Solution

renjith_nair
Legend

it shouldn't matter. Here is a run anywhere example with multiple events with different number of URL

| makeresults 
| eval _raw="here are some url and http://firsturl.com and some text again url http://www.secondurl.com and again some text URL http://www.third.com/" 
| append 
    [| makeresults 
    | eval _raw="here are some url and http://fourth.com and some text again url"] 
| append 
    [| makeresults 
    | eval _raw="here are some url and http://fifth.com and some text again url and some text again url http://www.sixth.com and again some text http://www.seventh.com and some http://www.moreandmore.com"] 
| rex max_match=0 "\b(?<domain2>(?:http?://|www\.)(?:[0-9a-z-]+\.)+[a-z]{2,63})/?"
---
What goes around comes around. If it helps, hit it with Karma 🙂

View solution in original post

CyberWolf
Path Finder

its a good idea, but in my case i don't know what domains they will be in the _raw, so i cant predict the list. Some events have one domain and it captured but the next event has 5, next will have 12 and each event has different domains on the raw.

0 Karma

renjith_nair
Legend

it shouldn't matter. Here is a run anywhere example with multiple events with different number of URL

| makeresults 
| eval _raw="here are some url and http://firsturl.com and some text again url http://www.secondurl.com and again some text URL http://www.third.com/" 
| append 
    [| makeresults 
    | eval _raw="here are some url and http://fourth.com and some text again url"] 
| append 
    [| makeresults 
    | eval _raw="here are some url and http://fifth.com and some text again url and some text again url http://www.sixth.com and again some text http://www.seventh.com and some http://www.moreandmore.com"] 
| rex max_match=0 "\b(?<domain2>(?:http?://|www\.)(?:[0-9a-z-]+\.)+[a-z]{2,63})/?"
---
What goes around comes around. If it helps, hit it with Karma 🙂

CyberWolf
Path Finder

Yap it makes since now, it worked thanks!

0 Karma

renjith_nair
Legend

Try adding the max_match parameter

Here is a run anywhere example.  Please note that the resulted field from  max_match will be a multivalued field in case you want further operation on the field.

| makeresults
| eval _raw="here are some url and http://firsturl.com and some text again url http://www.secondurl.com and again some text URL http://www.third.com/"
| rex max_match=0 "\b(?<domain2>(?:http?://|www\.)(?:[0-9a-z-]+\.)+[a-z]{2,63})/?"

 

---
What goes around comes around. If it helps, hit it with Karma 🙂
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

What Is Splunk? Here’s What You Can Do with Splunk

Hey Splunk Community, we know you know Splunk. You likely leverage its unparalleled ability to ingest, index, ...

Level Up Your .conf25: Splunk Arcade Comes to Boston

With .conf25 right around the corner in Boston, there’s a lot to look forward to — inspiring keynotes, ...

Manual Instrumentation with Splunk Observability Cloud: How to Instrument Frontend ...

Although it might seem daunting, as we’ve seen in this series, manual instrumentation can be straightforward ...