Splunk Search

How to search and count all events from referrers (hostnames) that were not present in yesterday's events?

jonarnes
Engager

Hi. I am trying to understand how I can list new referrers (hostnames) :
rex field=headers.Referer "^https?://(www.)?(?[^/]+)"| search refer_domain="*"
compared to yesterday. That means, I want to list (and count) all events from a referrer that was not present yesterday. Possible?

0 Karma

chris
Motivator

Hi,
I'm not sure whether the rex you have in the question works so I changed it. What you can do:

  • is a base search on the events that have the headers.Referer field over the last 2 days,
  • extract the referer domain (ref_domain).
  • the dc function for stats will count how many different day values have occurred for
  • then you create a field that contains the current weekday
  • then you search for the ref_domains that only occur on one day and whose day matches today

    earliest=-1d@d | rex field=headers.Referer "^https?://?(www.)?(?[^/]+)" | stats dc(date_wday) as count by ref_domain | stats dc(date_wday) as count values(date_wday) as day by ref_domain | eval today=now() | convert timeformat="%A" ctime(today) AS today | eval today=lower(today) | eval match=if(day==today,"ok","nok") | search count=1 match=ok

Regards
Chris

0 Karma

chris
Motivator

This is not very elegant go with martin_mueller's answer

martin_mueller
SplunkTrust
SplunkTrust

Run this over the two days in question, e.g. -2d@d to @d:

  base search yielding some "referer" field | stats count earliest(_time) as earliesttime by referer
| addinfo | where earliesttime >= relative_time(info_min_time, "+d") | fields - info* earliesttime

This counts over two days and gets you the timestamp of the earliest event. If that event is within a day from the start of the timerange, e.g. happened during the first of the two days, that referer is discarded. What remains has appeared on the second day but not on the first along with a count for that second day.

Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...