Error in Splunk Security Essentials with macro `ut_parse_extended (url, list)`

[socverne]

Good Morning.

I am using the "Splunk Security Essentials" add-on and when executing a search, I get an error in the macro ut_parse_extended (url, list). I put the search:

"index = * sourcetype = pan: threat OR (tag = web tag = proxy) earliest = -20m @ m earliest = -5m @ m | eval list =" mozilla "|ut_parse_extended (url, list)| lookup dynamic_dns_lookup domain as ut_domain OUTPUT inlist | search inlist = true | table _time ut_domain inlist bytes * uri "

And the error:

"Error in 'SearchParser': The search specifies a macro 'ut_parse_extended' that can not be found. Reasons include: the macro name is misspelled, you do not have" read "permission for the macro, or the macro has not been shared with this application, Click Settings, Advanced search, Search Macros to view macro information. "

Could you help me? Thank you.

A greeting.

[MuS]

Hi socverne,

The docs of the apps https://splunkbase.splunk.com/app/3435/#/details tell you this:

 The searches rely on tools included in Splunk platform to perform anomaly detection, such as the URL toolbox to detect Shannon entropy in URLs

So you need to install https://splunkbase.splunk.com/app/2734/ to be able to use the macro.

BUT, and some greeting to the BOTS 2019 Team here ;), be aware that this macro returns wrong 2nd level domains for some URL's!

The only way to get around this is to actually use a regex and get the 2nd level domains this way. I ended up with this regex to get the correct 2nd level domains:

 (?<my2ndLevelDomain>[^.]+)\.(?:(?:com|net|org|edu|gov|asn|id|csiro|)\.au|co\.(?:bb|ck|cr|in|id|il|jp|nz|za|kr|th|uk)|[\w\s]{2,})$

Hope this helps ...

cheers, MuS

PS: @eliasit, yes you still can install the app on Splunk 7.2.x and use it with the above mentioned issues/problems 😉

[HiroshiSatoh]

Who created this macro(ut_parse_extended )? Please check the permission of this macro.

[socverne]

Thanks for your answer. I have obscured it, and the macro does not exist. With what content can I create it? Thank you.
A greeting.

[HiroshiSatoh]

ut_parse_extended is in "URL Toolbox". Please install and use APP.

https://splunkbase.splunk.com/app/2734/#/details

[eliasit]

"URL Toolbox" is not listed as compatible with Splunk 7.2 or higher. Is there an alternative? Can "URL Toolbox" be manually install without causing issues? Or could the macro be extracted from the install file?