Hello splunkfans,
i'm kind of running out of ideas and this is my first contact to streamstats. 😕
I am working on a statistic of botnet portscans on my firewalllogs. The goal is based on the firewallevents on a specific interface and direction, to show how many different public IPs have scanned how many dest_ports, events=attacks, and in what timerange.
The break where i assume it is a botnet portscan is when i get over 150 events in a 2 min flow.
Here is my search:
index="security" interface=igb0 direction=in | streamstats time_window=2min count(_raw) as "attacks" distinct_count(source_ip) as "attackers", distinct_count(destination_port) as "attacked ports" | search attacks>150
What i am looking at due to my search is a nice timeline over the last 3 days with 3 peaks that represent the time_window=2min of my streamstats.
My goal is just to get a table of these peaks and the timerange they occured.
Like this: table attacks, attackers, "attacked ports", //"timerange first and last event"//
Problem i have is that this table without time (as i have no solution for that), shows me all streamstats events like this:
attacks attackers attacked ports
151 34 9
152 34 9
153 34 9
154 34 10
155 34 10
156 34 10
157 34 10
Searchresult of just the streamstats is of cause the events themselfs that are relevant in the time_window.
How can i get just the peaks of this and the timerange between the first and last event of these?
I tried so many combos with max() and top, but as i dont know how many peaks will occure, i cant regulate the top.
A timechart seams to be a dirty solution with a span=2min but a portscan can happen between 00:01:45 and 00:02:15 and would be split in half and not recognised.
Anyone can guide me in the right direction on how to get the tops of this stream? 🙂
If you are looking for just the peaks, and not the individual events, then you probably want to use timechart instead.
index="security" interface=igb0 direction=in
| timechart span=2min count as "attacks", dc(source_ip) as "attackers", dc(destination_port) as "attacked ports", min(_time) as "start Time", max(_time) as "End Time" | search attacks>150
fix formatting
Try using first() and last() statistical functions:
index="security" interface=igb0 direction=in source_ip=* destination_port=*
| streamstats time_window=2m count as "attacks" dc(source_ip) as "attackers", dc(destination_port) as "attacked ports" first(_time) as FirstTime last(_time) as LastTime
| search attacks>150
| eval FirstTime=strftime(FirstTime,"%c")
| eval LastTime=strftime(LastTime,"%c")
| table attacks attackers "attacked ports" FirstTime LastTime
PS: I have added source_ip=* and destination_port=* assuming they are always present. If not remove that from the base search. I have used strftime to convert to String Date Format.
We are getting close, thanks for the replies!
I tried this like niketnilay answered with modifications. It is very close, just have to get rid of the overlaps, see below.
index="security" interface=igb0 direction=in
| streamstats time_window=2min count as "attacks" dc(source_ip) as "attackers", dc(destination_port) as "attacked ports" first(_time) as FirstTime last(_time) as LastTime
| search attacks>150
| eval FirstTime=strftime(FirstTime,"%c")
| eval LastTime=strftime(LastTime,"%c")
| table attacks attackers "attacked ports" FirstTime LastTime
| sort -attacks
| dedup FirstTime
Got me this:
attacks attackers attacked ports FirstTime LastTime
902 90 106 Tue Jan 3 16:34:06 2017 Tue Jan 3 16:32:12 2017
693 69 81 Tue Jan 3 00:15:08 2017 Tue Jan 3 00:13:19 2017
691 68 79 Tue Jan 3 00:14:26 2017 Tue Jan 3 00:12:30 2017
565 91 22 Tue Jan 3 16:34:31 2017 Tue Jan 3 16:32:34 2017
426 70 14 Tue Jan 3 00:15:41 2017 Tue Jan 3 00:13:42 2017
371 70 13 Tue Jan 3 00:15:46 2017 Tue Jan 3 00:13:47 2017
339 34 44 Wed Jan 4 23:23:57 2017 Wed Jan 4 23:22:16 2017
264 33 41 Wed Jan 4 23:23:03 2017 Wed Jan 4 23:21:04 2017
262 91 19 Tue Jan 3 16:34:54 2017 Tue Jan 3 16:32:55 2017
248 34 41 Wed Jan 4 23:22:59 2017 Wed Jan 4 23:21:01 2017
239 35 12 Wed Jan 4 23:24:32 2017 Wed Jan 4 23:22:33 2017
202 35 41 Wed Jan 4 23:22:50 2017 Wed Jan 4 23:20:51 2017
Yes DalJeanis i'm just looking for the peaks, but a timechart isn't accurate if the attack is cut into two time spans. Am i wrong?
I'm wondering why these dont have the same firsttime... There are overlaps in the timeranges. This is due to the search attacks>150 after the streamline.
Any chance to get this into the stream as a condition?