How to search and count all events from referrers (hostnames) that were not present in yesterday's events?

[jonarnes]

Hi. I am trying to understand how I can list new referrers (hostnames) :
rex field=headers.Referer "^https?://(www.)?(?[^/]+)"| search refer_domain="*"
compared to yesterday. That means, I want to list (and count) all events from a referrer that was not present yesterday. Possible?

[chris]

Hi,
I'm not sure whether the rex you have in the question works so I changed it. What you can do:

• is a base search on the events that have the headers.Referer field over the last 2 days,
• extract the referer domain (ref_domain).
• the dc function for stats will count how many different day values have occurred for
• then you create a field that contains the current weekday

• then you search for the ref_domains that only occur on one day and whose day matches today

  earliest=-1d@d | rex field=headers.Referer "^https?://?(www.)?(?[^/]+)" | stats dc(date_wday) as count by ref_domain | stats dc(date_wday) as count values(date_wday) as day by ref_domain | eval today=now() | convert timeformat="%A" ctime(today) AS today | eval today=lower(today) | eval match=if(day==today,"ok","nok") | search count=1 match=ok

Regards
Chris

[chris]

This is not very elegant go with martin_mueller's answer

[martin_mueller]

Run this over the two days in question, e.g. -2d@d to @d:

  base search yielding some "referer" field | stats count earliest(_time) as earliesttime by referer
| addinfo | where earliesttime >= relative_time(info_min_time, "+d") | fields - info* earliesttime

This counts over two days and gets you the timestamp of the earliest event. If that event is within a day from the start of the timerange, e.g. happened during the first of the two days, that referer is discarded. What remains has appeared on the second day but not on the first along with a count for that second day.