Getting Data In

Extracting domain names (mvindex?)

howyagoin
Contributor

I get the feeling this is going to be a tough one to solve, but, I'm trying to aggregate results of a search based upon domain name. I realise that this is a bit of a non-starter simply because of things like fubar.com versus fubar.co.uk, but, my first approach to this was:

search term | eval mydomain=split(dest_host,".") | eval tld=mvindex(mydomain,-1) | eval target=mvindex(mydomain,1) | eval hoster=target.".".tld 

And this works most of the time, but not all of the time.

I'm operating on the (questionable) assumption that the last two elements split by dest_host are likely to be the domain name - but maybe there's a better way to perform this aggregation. I'm trying to group together results which might be for host123-ab.fubar.com and host445-qx.fubar.com under fubar.com, for example.

I suppose another way to do this is to use some sort of a lookup table with all well-known TLDs and major sub-domains (.co.uk, .ac.uk and so forth) -- but it feels like a problem others must have tried to resolve here already.

Suggestions welcome!

Tags (2)

woodcock
Esteemed Legend

There are apps for this:

URL Parser: https://splunkbase.splunk.com/app/1545/
URL Toolbox: https://splunkbase.splunk.com/app/2734/

0 Karma
Get Updates on the Splunk Community!

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...