All Apps and Add-ons

URL Toolbox ut_parse_extended_lookup not working

browncardigan
Path Finder

Running the search with the long-form ut_parse_extended_lookup command:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| lookup ut_parse_extended_lookup url AS query 
| stats count by ut_domain

Fails to identify obvious domains like foxsports.com.au, instead breaking it out as ut_domain=com.au and ut_subdomain_parts=foxsports.

Meanwhile, running (what should be) the same search by using the macro:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| `ut_parse(query,list)`
| stats count by ut_domain

parses the same domains perfectly, this time as ut_domain=foxsports.com.au

I checked suffix_list_mozilla.dat and confirmed the com.au 2LD entry exists, and I can't identify any meaningful differences between the manual query I ran and what's abstracted by the macro is macros.conf.

[ut_parse_extended(2)]
args = url, list
definition = lookup ut_parse_extended_lookup url as $url$ list as $list$ | spath input=ut_subdomain_parts | fields - ut_subdomain_parts

Does anyone know why I'm getting such poor results from running the long-form version of the command, as opposed to the macro?

0 Karma
1 Solution

to4kawa
Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

View solution in original post

0 Karma

to4kawa
Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

0 Karma

browncardigan
Path Finder

The spath argument just removes the ut_subdomain_parts field, but your response did highlight the fact I didn't add list AS list to the lookup command - adding this solved the issue as it uses the iana list otherwise.

I'd tried this before but instead did list AS mozilla so it didn't work.

Thanks for your response!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...