Splunk Search

How to search and count all events from referrers (hostnames) that were not present in yesterday's events?

jonarnes
Engager

Hi. I am trying to understand how I can list new referrers (hostnames) :
rex field=headers.Referer "^https?://(www.)?(?[^/]+)"| search refer_domain="*"
compared to yesterday. That means, I want to list (and count) all events from a referrer that was not present yesterday. Possible?

0 Karma

chris
Motivator

Hi,
I'm not sure whether the rex you have in the question works so I changed it. What you can do:

  • is a base search on the events that have the headers.Referer field over the last 2 days,
  • extract the referer domain (ref_domain).
  • the dc function for stats will count how many different day values have occurred for
  • then you create a field that contains the current weekday
  • then you search for the ref_domains that only occur on one day and whose day matches today

    earliest=-1d@d | rex field=headers.Referer "^https?://?(www.)?(?[^/]+)" | stats dc(date_wday) as count by ref_domain | stats dc(date_wday) as count values(date_wday) as day by ref_domain | eval today=now() | convert timeformat="%A" ctime(today) AS today | eval today=lower(today) | eval match=if(day==today,"ok","nok") | search count=1 match=ok

Regards
Chris

0 Karma

chris
Motivator

This is not very elegant go with martin_mueller's answer

martin_mueller
SplunkTrust
SplunkTrust

Run this over the two days in question, e.g. -2d@d to @d:

  base search yielding some "referer" field | stats count earliest(_time) as earliesttime by referer
| addinfo | where earliesttime >= relative_time(info_min_time, "+d") | fields - info* earliesttime

This counts over two days and gets you the timestamp of the earliest event. If that event is within a day from the start of the timerange, e.g. happened during the first of the two days, that referer is discarded. What remains has appeared on the second day but not on the first along with a count for that second day.

Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...