Web Data Model no referrer

[jacqu3sy]

Can anyone help with the following please. Im looking to run a tstats query against the Web Data Model but exclude results where a user has been referrered to a destination.

If I run this directly against the index, it runs fine, but returns nothing when running it against the DM.

Query:

| tstats summariesonly count from datamodel=Web where Web.dest=kali AND Web.http_method=POST NOT Web.category="Web Ads/Analytics" NOT Web.http_user_agent="Mozilla/5.0 Zoom.App" NOT Web.http_referrer="*"

The other NOT statements work fine, but the NOT against Web.http_referrer just returns no results. I have tried using != instead of the NOT but I get the same problem.

Any ideas?

[sheamus69]

Looking at your query, I think the issue lies with the fact the the http_referrer field should be populated with something, even if it is "unknown", which I believe is what the datamodel autopopulates into an empty field. The code below should fix that?

| tstats summariesonly count from datamodel=Web where Web.dest=kali AND Web.http_method=POST NOT Web.category="Web Ads/Analytics" NOT Web.http_user_agent="Mozilla/5.0 Zoom.App" NOT Web.http_referrer="unknown"

[jkat54]

Yeah you could use this search to see too

| tstats values(http_referrer) from datamodel=Web

[sheamus69]

Wouldn't that be

| tstats values(Web.http_referrer) from datamodel=Web

?

[jkat54]

Sure 🙂 ... my phone really doesn’t like it when I try typing that though. Was hoping someone would comment like you did.

[sheamus69]

As a tangientially related query, any idea what generates that particular user agent?

[jkat54]

Is the datamodel accelerated? If not, tstats wont work because it only works on indexed fields within tsidx files.

[jacqu3sy]

yes its accelerated. It works fine. The only problem is excluding from the results URL's that people have been referred to.

I only want to see when people have browsed straight to the site in question rather than referrals which produces too much noise.