All Apps and Add-ons

URL Toolbox ut_parse_extended_lookup not working

browncardigan
Path Finder

Running the search with the long-form ut_parse_extended_lookup command:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| lookup ut_parse_extended_lookup url AS query 
| stats count by ut_domain

Fails to identify obvious domains like foxsports.com.au, instead breaking it out as ut_domain=com.au and ut_subdomain_parts=foxsports.

Meanwhile, running (what should be) the same search by using the macro:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| `ut_parse(query,list)`
| stats count by ut_domain

parses the same domains perfectly, this time as ut_domain=foxsports.com.au

I checked suffix_list_mozilla.dat and confirmed the com.au 2LD entry exists, and I can't identify any meaningful differences between the manual query I ran and what's abstracted by the macro is macros.conf.

[ut_parse_extended(2)]
args = url, list
definition = lookup ut_parse_extended_lookup url as $url$ list as $list$ | spath input=ut_subdomain_parts | fields - ut_subdomain_parts

Does anyone know why I'm getting such poor results from running the long-form version of the command, as opposed to the macro?

0 Karma
1 Solution

to4kawa
Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

View solution in original post

0 Karma

to4kawa
Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

0 Karma

browncardigan
Path Finder

The spath argument just removes the ut_subdomain_parts field, but your response did highlight the fact I didn't add list AS list to the lookup command - adding this solved the issue as it uses the iana list otherwise.

I'd tried this before but instead did list AS mozilla so it didn't work.

Thanks for your response!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...