Splunk Search

How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

ng87
Path Finder

i am trying to think of a way to craft a search that will look for any hosts doing web-requests to the same site/url at regular the same intervals.
Basic idea is that Host A does a request to WebsiteA every X amount of seconds/minutes (if i could add a range like every 15-20 seconds that would be even better due to timing of logs etc.. ).

Any ideas on how to do this in splunk?

0 Karma
1 Solution

aweitzman
Motivator

Group your items by Host and Website and get time deltas on them by using streamstatsacross them with a window encompassing just the previous item (size of 2), and using global=f to ensure that the time deltas are by group:

...[original search]... | streamstats window=2 global=f range(_time) as timedelta by Host Website

Then, remove the 0's (which all of the last entries for each Host/Website combo will have) and do some statistics:

... 
| where timedelta > 0 
| stats avg(timedelta) as DeltaAvg range(timedelta) as DeltaRange by Host Website 
| table Host Website DeltaAvg DeltaRange

Then filter to what you need beyond that.

View solution in original post

stephane_cyrill
Builder

Hi to add a range of time,try with the following commands:

span=.......s

    OR

per_second( .....)

aweitzman
Motivator

Group your items by Host and Website and get time deltas on them by using streamstatsacross them with a window encompassing just the previous item (size of 2), and using global=f to ensure that the time deltas are by group:

...[original search]... | streamstats window=2 global=f range(_time) as timedelta by Host Website

Then, remove the 0's (which all of the last entries for each Host/Website combo will have) and do some statistics:

... 
| where timedelta > 0 
| stats avg(timedelta) as DeltaAvg range(timedelta) as DeltaRange by Host Website 
| table Host Website DeltaAvg DeltaRange

Then filter to what you need beyond that.

View solution in original post

ng87
Path Finder

thanks a lot, looks promising, will give that a go tomorrow.
could i increase the window to a lot more than 2? ( as 2 will give me loads of results where something more like 10-15 will really filter it down to what i am looking for )

0 Karma

aweitzman
Motivator

I'm not sure what you mean.

  1. streamstats only allows you to perform aggregate operations on the items in your window, so if you had more than two events in the window, there's no operation you could use to determine the time delta between each event, which is what you're looking for. You need to ensure that the events are consecutive (within the group), and then you can use the range operation to get what you need.
  2. By performing the stats command after the streamstats you are reducing the number of results for each Host-Website combination to 1, so you shouldn't be overly burdened with results.

However, if what you care about is that you are getting too many Host-Website combinations, and only care about ones that happen relatively frequently, then what you want to do is add a stats that just does a count in the group, and then filter out smaller counts:

...[original search]... 
| streamstats window=2 global=f range(_time) as timedelta by Host Website
| where timedelta > 0
| stats count as n avg(timedelta) as DeltaAvg range(timedelta) as DeltaRange by Host Website
| table Host Website n DeltaAvg DeltaRange
| where n>10
0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!