Splunk Search

How to chart over time based on small daily samples

NathanaClarke
New Member

Hi,

I'm relatively new to creating splunk reports and simple ones have been easy but now I need a shortcut/help to get this done while I am still learning 😉

I am trying to create a report based on one type of event (firewall denies), by policy (the firewall deny group). I have a lot of these types of events coming in from multiple firewalls. I'd like to schedule a report that averages just the previous day's events per firewall deny policy and adds that average to a rolling chart that shows the last 30 days. I'm cognisant of keeping the effort my splunk host has to expend to a minimum so I only want to eval the previous day's events and just add that to a chart that I can download or share.

Is that doable?

Tags (1)
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

I agree with @niketnilay that in the long run you should probably be looking to create a summary index to keep your daily stats.

I am not certain, though, what you mean by "average" the prior day's events. Normally, you would be looking to sum them up and keep them individually, so you can review trends and details. Also, the word "sample" in the title worries me, because you probably want the entire day's data, not a sample.

Also, before you go to the trouble of making a summary index, you might consider starting with just a csv file.

To create it, you would run something like this, once...

earliest=-30d@d latest=@d ...Your search that selects the events you want...
| bin _time span=1d
| stats count as ViolationCount by FirewallName PolicyName _time
| outputcsv myViolationCounts.csv

... you can run it in chunks with append=t if it would take too long as a single run.


Thereafter, daily, you'd run the same thing with earliest=-1d@d, and read in the older file to kill any dates that were older than 30 days. (Killing the data is entirely optional... you could keep it in the file and filter it out before presentation, which would give you the ability to present a longer-term view of the trends, for example a sparkline of the last twelve weeks, etc.)

earliest=-1d@d latest=@d ...Your search that selects the events you want...
| bin _time span=1d
| stats count as ViolationCount by FirewallName PolicyName _time
| inputcsv append=t myViolationCounts.csv
| where _time>= relative_time(now(),"-30d@d")
| outputcsv append=f myViolationCounts.csv

After running the above, you can run your report that produces a visualization based on the data. (This could also be done at the end of the above, but it would make reruns problematic, since they might potentially double the count for the day the report was run twice.)

You can subset this information various ways, based upon your business usage.

Overall violations:

 | inputcsv append=t myViolationCounts.csv 
 | timechart span=1d sum(ViolationCount) as count 

Average violations per policy

 | inputcsv append=t myViolationCounts.csv 
 | timechart span=1d avg(ViolationCount) 

Thirty days history of the Top 3 policies violated today

 | inputcsv append=t myViolationCounts.csv 
 | append 
    [| inputcsv append=t myViolationCounts.csv 
     | eventstats max(_time) as maxtime 
     | where _time = maxtime 
     | sort 3 - ViolationCount 
     | table PolicyName 
     | eval Top3="True"
    ]
 | eventstats max(Top3) as Top3 by PolicyName
 | where Top3="True"
 | fields - Top3
 | timechart span=1d sum(ViolationCount) as count by PolicyName

Those are just some samples of the information that you could quickly pull from a csv or a summary index of this information.

Happy splunking!

0 Karma

niketn
Legend

You can summarize your daily stats using summary indexing. One of the methods is to use collect command and save as a daily scheduled search. Refer to documentation: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Collect

Here is a short video that you can use to grasp the concept: https://www.splunk.com/view/SP-CAAACZW

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...