topic Re: Web Data Model no referrer in Splunk Search

Web Data Model no referrer

jacqu3sy — Tue, 29 Sep 2020 14:49:18 GMT

Can anyone help with the following please. Im looking to run a tstats query against the Web Data Model but exclude results where a user has been referrered to a destination.

If I run this directly against the index, it runs fine, but returns nothing when running it against the DM.

Query:

| tstats summariesonly count from datamodel=Web where Web.dest=kali AND Web.http_method=POST NOT Web.category="Web Ads/Analytics" NOT Web.http_user_agent="Mozilla/5.0 Zoom.App" NOT Web.http_referrer="*"

The other NOT statements work fine, but the NOT against Web.http_referrer just returns no results. I have tried using != instead of the NOT but I get the same problem.

Any ideas?

Re: Web Data Model no referrer

jkat54 — Mon, 10 Jul 2017 10:02:16 GMT

Is the datamodel accelerated? If not, tstats wont work because it only works on indexed fields within tsidx files.

Re: Web Data Model no referrer

jacqu3sy — Mon, 10 Jul 2017 10:10:54 GMT

yes its accelerated. It works fine. The only problem is excluding from the results URL's that people have been referred to.

I only want to see when people have browsed straight to the site in question rather than referrals which produces too much noise.

Re: Web Data Model no referrer

sheamus69 — Tue, 20 Feb 2018 11:05:02 GMT

As a tangientially related query, any idea what generates that particular user agent?

Re: Web Data Model no referrer

sheamus69 — Tue, 20 Feb 2018 11:14:35 GMT

Looking at your query, I think the issue lies with the fact the the http_referrer field should be populated with something, even if it is "unknown", which I believe is what the datamodel autopopulates into an empty field. The code below should fix that?

| tstats summariesonly count from datamodel=Web where Web.dest=kali AND Web.http_method=POST NOT Web.category="Web Ads/Analytics" NOT Web.http_user_agent="Mozilla/5.0 Zoom.App" NOT Web.http_referrer="unknown"

Re: Web Data Model no referrer

jkat54 — Tue, 20 Feb 2018 13:31:26 GMT

Yeah you could use this search to see too

| tstats values(http_referrer) from datamodel=Web

Re: Web Data Model no referrer

sheamus69 — Tue, 20 Feb 2018 14:05:28 GMT

Wouldn't that be

| tstats values(Web.http_referrer) from datamodel=Web

Re: Web Data Model no referrer

jkat54 — Tue, 20 Feb 2018 14:29:46 GMT

Sure 🙂 ... my phone really doesn’t like it when I try typing that though. Was hoping someone would comment like you did.