All Apps and Add-ons

URL Toolbox ut_parse_extended_lookup not working

Path Finder

Running the search with the long-form utparseextended_lookup command:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| lookup ut_parse_extended_lookup url AS query 
| stats count by ut_domain

Fails to identify obvious domains like foxsports.com.au, instead breaking it out as ut_domain=com.au and utsubdomainparts=foxsports.

Meanwhile, running (what should be) the same search by using the macro:

sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
| eval list="mozilla" 
| `ut_parse(query,list)`
| stats count by ut_domain

parses the same domains perfectly, this time as ut_domain=foxsports.com.au

I checked suffixlistmozilla.dat and confirmed the com.au 2LD entry exists, and I can't identify any meaningful differences between the manual query I ran and what's abstracted by the macro is macros.conf.

[ut_parse_extended(2)]
args = url, list
definition = lookup ut_parse_extended_lookup url as $url$ list as $list$ | spath input=ut_subdomain_parts | fields - ut_subdomain_parts

Does anyone know why I'm getting such poor results from running the long-form version of the command, as opposed to the macro?

0 Karma
1 Solution

Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

View solution in original post

0 Karma

Ultra Champion
 sourcetype=infoblox:dns record_type IN (TXT,A,AAAA) 
 | eval list="mozilla" 
 | lookup ut_parse_extended_lookup url AS query list AS list
 | spath input=ut_subdomain_parts 
 | fields - ut_subdomain_parts
 | stats count by ut_domain

Hi, How about adding spath?

View solution in original post

0 Karma

Path Finder

The spath argument just removes the utsubdomainparts field, but your response did highlight the fact I didn't add list AS list to the lookup command - adding this solved the issue as it uses the iana list otherwise.

I'd tried this before but instead did list AS mozilla so it didn't work.

Thanks for your response!

0 Karma