Knowledge Management

How do you do URL domain analysis with the Web datamodel?

MonkeyK
Builder

I wold like to count URL domains for sites categorized as phishing or malware. The closest that I know how to do this is to look at url, but this can frequently be uniquified in the age of REST.

So how does one go about getting accelerated query results for URL domains?

0 Karma
1 Solution

MonkeyK
Builder

Actually, looks like the attribute "site" addresses domain, so I guess that all I needed was

|tstats count FROM datamodel=Web.Web WHERE Web.category=malware BY Web.site | sort -count +Web.site

still some confusion around the eval. I guess I'll have to take that one up in a different question

View solution in original post

0 Karma

MonkeyK
Builder

Actually, looks like the attribute "site" addresses domain, so I guess that all I needed was

|tstats count FROM datamodel=Web.Web WHERE Web.category=malware BY Web.site | sort -count +Web.site

still some confusion around the eval. I guess I'll have to take that one up in a different question

View solution in original post

0 Karma

MonkeyK
Builder

So, for example I have this

|tstats summariesonly count FROM datamodel=Web.Web WHERE Web.category=malware BY Web.url

and want to summarize by domain instead of URL. One thought that I had was to do some sort of eval on Web.url and then sum the counts, but I cannot even get eval to work

|tstats summariesonly count FROM datamodel=Web.Web WHERE Web.category=malware BY Web.url
| eval urlDom=Web.url
| stats sum(count) by> urlDom

returns nothing because urlDom is not evaluated. I can see this by trying

|tstats summariesonly count FROM datamodel=Web.Web WHERE Web.category=malware BY Web.url
| eval urlDom=Web.url | fields urlDom, Web.url, count
I get back empty values for urlDom.
Is this an error?

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!