How to chart over time based on small daily samples

[NathanaClarke]

Hi,

I'm relatively new to creating splunk reports and simple ones have been easy but now I need a shortcut/help to get this done while I am still learning 😉

I am trying to create a report based on one type of event (firewall denies), by policy (the firewall deny group). I have a lot of these types of events coming in from multiple firewalls. I'd like to schedule a report that averages just the previous day's events per firewall deny policy and adds that average to a rolling chart that shows the last 30 days. I'm cognisant of keeping the effort my splunk host has to expend to a minimum so I only want to eval the previous day's events and just add that to a chart that I can download or share.

Is that doable?

[DalJeanis]

I agree with @niketnilay that in the long run you should probably be looking to create a summary index to keep your daily stats.

I am not certain, though, what you mean by "average" the prior day's events. Normally, you would be looking to sum them up and keep them individually, so you can review trends and details. Also, the word "sample" in the title worries me, because you probably want the entire day's data, not a sample.

Also, before you go to the trouble of making a summary index, you might consider starting with just a csv file.

To create it, you would run something like this, once...

earliest=-30d@d latest=@d ...Your search that selects the events you want...
| bin _time span=1d
| stats count as ViolationCount by FirewallName PolicyName _time
| outputcsv myViolationCounts.csv

... you can run it in chunks with append=t if it would take too long as a single run.

---

Thereafter, daily, you'd run the same thing with earliest=-1d@d, and read in the older file to kill any dates that were older than 30 days. (Killing the data is entirely optional... you could keep it in the file and filter it out before presentation, which would give you the ability to present a longer-term view of the trends, for example a sparkline of the last twelve weeks, etc.)

earliest=-1d@d latest=@d ...Your search that selects the events you want...
| bin _time span=1d
| stats count as ViolationCount by FirewallName PolicyName _time
| inputcsv append=t myViolationCounts.csv
| where _time>= relative_time(now(),"-30d@d")
| outputcsv append=f myViolationCounts.csv

After running the above, you can run your report that produces a visualization based on the data. (This could also be done at the end of the above, but it would make reruns problematic, since they might potentially double the count for the day the report was run twice.)

You can subset this information various ways, based upon your business usage.

Overall violations:

 | inputcsv append=t myViolationCounts.csv 
 | timechart span=1d sum(ViolationCount) as count 

Average violations per policy

 | inputcsv append=t myViolationCounts.csv 
 | timechart span=1d avg(ViolationCount) 

Thirty days history of the Top 3 policies violated today

 | inputcsv append=t myViolationCounts.csv 
 | append 
    [| inputcsv append=t myViolationCounts.csv 
     | eventstats max(_time) as maxtime 
     | where _time = maxtime 
     | sort 3 - ViolationCount 
     | table PolicyName 
     | eval Top3="True"
    ]
 | eventstats max(Top3) as Top3 by PolicyName
 | where Top3="True"
 | fields - Top3
 | timechart span=1d sum(ViolationCount) as count by PolicyName

Those are just some samples of the information that you could quickly pull from a csv or a summary index of this information.

Happy splunking!

[niketn]

You can summarize your daily stats using summary indexing. One of the methods is to use collect command and save as a daily scheduled search. Refer to documentation: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Collect

Here is a short video that you can use to grasp the concept: https://www.splunk.com/view/SP-CAAACZW