Extracting domain names (mvindex?)

howyagoin — Sat, 14 Apr 2012 05:56:11 GMT

I get the feeling this is going to be a tough one to solve, but, I'm trying to aggregate results of a search based upon domain name. I realise that this is a bit of a non-starter simply because of things like fubar.com versus fubar.co.uk, but, my first approach to this was:

search term | eval mydomain=split(dest_host,".") | eval tld=mvindex(mydomain,-1) | eval target=mvindex(mydomain,1) | eval hoster=target.".".tld

And this works most of the time, but not all of the time.

I'm operating on the (questionable) assumption that the last two elements split by dest_host are likely to be the domain name - but maybe there's a better way to perform this aggregation. I'm trying to group together results which might be for host123-ab.fubar.com and host445-qx.fubar.com under fubar.com, for example.

I suppose another way to do this is to use some sort of a lookup table with all well-known TLDs and major sub-domains (.co.uk, .ac.uk and so forth) -- but it feels like a problem others must have tried to resolve here already.

Suggestions welcome!

Re: Extracting domain names (mvindex?)

woodcock — Sun, 05 Jul 2015 14:00:51 GMT

There are apps for this:

URL Parser: https://splunkbase.splunk.com/app/1545/
URL Toolbox: https://splunkbase.splunk.com/app/2734/

topic Re: Extracting domain names (mvindex?) in Getting Data In

Extracting domain names (mvindex?)

Re: Extracting domain names (mvindex?)