topic Re: How to write a search to find hosts that perform web requests to the same site/url at an exact interval? in Splunk Search

How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

ng87 — Mon, 20 Apr 2015 15:41:09 GMT

i am trying to think of a way to craft a search that will look for any hosts doing web-requests to the same site/url at regular the same intervals.
Basic idea is that Host A does a request to WebsiteA every X amount of seconds/minutes (if i could add a range like every 15-20 seconds that would be even better due to timing of logs etc.. ).

Any ideas on how to do this in splunk?

Re: How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

aweitzman — Mon, 20 Apr 2015 17:31:05 GMT

Group your items by Host and Website and get time deltas on them by using streamstatsacross them with a window encompassing just the previous item (size of 2), and using global=f to ensure that the time deltas are by group:

...[original search]... | streamstats window=2 global=f range(_time) as timedelta by Host Website

Then, remove the 0's (which all of the last entries for each Host/Website combo will have) and do some statistics:

... 
| where timedelta > 0 
| stats avg(timedelta) as DeltaAvg range(timedelta) as DeltaRange by Host Website 
| table Host Website DeltaAvg DeltaRange

Then filter to what you need beyond that.

Re: How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

ng87 — Mon, 20 Apr 2015 19:39:08 GMT

thanks a lot, looks promising, will give that a go tomorrow.
could i increase the window to a lot more than 2? ( as 2 will give me loads of results where something more like 10-15 will really filter it down to what i am looking for )

Re: How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

aweitzman — Mon, 20 Apr 2015 20:03:39 GMT

I'm not sure what you mean.

streamstats only allows you to perform aggregate operations on the items in your window, so if you had more than two events in the window, there's no operation you could use to determine the time delta between each event, which is what you're looking for. You need to ensure that the events are consecutive (within the group), and then you can use the range operation to get what you need.
By performing the stats command after the streamstats you are reducing the number of results for each Host-Website combination to 1, so you shouldn't be overly burdened with results.

However, if what you care about is that you are getting too many Host-Website combinations, and only care about ones that happen relatively frequently, then what you want to do is add a stats that just does a count in the group, and then filter out smaller counts:

...[original search]... 
| streamstats window=2 global=f range(_time) as timedelta by Host Website
| where timedelta > 0
| stats count as n avg(timedelta) as DeltaAvg range(timedelta) as DeltaRange by Host Website
| table Host Website n DeltaAvg DeltaRange
| where n>10

Re: How to write a search to find hosts that perform web requests to the same site/url at an exact interval?

stephane_cyrill — Mon, 20 Apr 2015 21:46:25 GMT

Hi to add a range of time,try with the following commands:

span=.......s

OR

per_second( .....)