Alerting

Splunk alert if today's volume is X percent different from past two weeks same day at same time is different

sahil237888
Path Finder

Hi Experts,

I need help in Splunk query to send alert if today's volume at latest time is X percent different from past two weeks volume same day at same time with 10 minutes bucket.

For dashboard thing, I have used below but not sure how to implement this in query and send alert as the below query is showing me whole day data at same time .

sourcetype=server_volume host=xxx earliest=-23h@h latest=@m
| eval ReportKey="Today"

| append [search sourcetype=server_volume host=xxx earliest=-23h@h-1w latest=@m-1w
| eval ReportKey="LastWeek"
| eval _time=_time+60*60*24*7]
| append [search sourcetype=server_volume host=xxx earliest=-23h@h-2w latest=@m-2w
| eval ReportKey="TwoWeeksBefore"
| eval _time=_time+60*60*24*14]
| timechart span=1m count(volume) by ReportKey

Tags (1)
0 Karma

rbechtold
Communicator

Hey Sahil,

Give the timewrap command a try!

Copy and paste this search into Splunk:

|gentimes start=-15 end=1 increment=1m
|rename starttime AS _time
| table _time 
| eval volume=random()%1000
| bin _time span=10m aligntime=@m
| chart sum(volume) AS sum_volume OVER _time
| sort - _time
| streamstats count
| eval count = count 
| eval volume = round(sum_volume - count,1)
| fields - sum_volume count
`comment("This is just a comment to let you know that you don't need to worry about anything above this line. It's just used for generating a dataset")`

| timewrap 1d
| table _time *latest_day *7days_before *14days_before
| head 1
| rename volume_latest_day AS Today, volume_7days_before AS LastWeek, volume_14days_before AS TwoWeeksBefore
| eval LastWeek_Percent_Change = round((LastWeek - Today) / Today * 100,2)
| eval TwoWeeksBefore_Percent_Change = round((TwoWeeksBefore - Today) / Today * 100,2)
| table _time Today LastWeek LastWeek_Percent_Change TwoWeeksBefore TwoWeeksBefore_Percent_Change

The search should find what your volume is today at this moment, and compare it to what the volume at both one week ago, and two weeks ago. To top it off, we'll calculate the percent difference from today for both last week, and two weeks ago!

Documentation for timewrap can be found here: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/timewrap

I hope this helps, let me know if you have any questions!

0 Karma

sahil237888
Path Finder

Where I can put sourcetype and my search string in the above query.

0 Karma

rbechtold
Communicator

Firstly: Make sure your time range picker is set to 15 days ago for this search to function
Click here to see what your time picker should look like

Also, what does your volume field look like? Judging from the name of the field, I am hesitant to recommend "count" as a metric for aggregating volume. For the sake of safetly, I'm going to use average in my search, but you can change it back if you'd like.

sourcetype=server_volume host=xxx 
| timechart span=10m avg(volume) AS volume
| timewrap 1d
| table _time *latest_day *7days_before *14days_before
| head 1
| rename volume_latest_day AS Today, volume_7days_before AS LastWeek, volume_14days_before AS TwoWeeksBefore
| eval LastWeek_Percent_Change = round((LastWeek - Today) / Today * 100,2)
| eval TwoWeeksBefore_Percent_Change = round((TwoWeeksBefore - Today) / Today * 100,2)
| table _time Today LastWeek LastWeek_Percent_Change TwoWeeksBefore TwoWeeksBefore_Percent_Change
0 Karma

nareshinsvu
Builder

Can you try this?

sourcetype=server_volume host=xxx
| timechart span=10m count(volume) by ReportKey as F |
timewrap 2w series=short |
sort -_time |
where _time>=relative_time(now(),"-10m@m") |
eval Time=strftime(_time,"%D %H:%M")|
rename F_s1 as 2_weeks_ago |
rename F_s0 as Today |
table Time Today 2_weeks_ago

0 Karma

sahil237888
Path Finder

I tried the same but it didn't worked.
It is giving me error "Error in 'timechart' command: The argument 'as' is invalid."

is it possible to get output like :
Today's last 10 minutes each minute wise and then compare with past week's same day same time minute wise comparison

0 Karma

nareshinsvu
Builder

@sahil237888 - It is working on my datamodels.

I think you have to define your datamodel/table and then run this query as volume and ReportKey should be a field names?

| from datamodel:"your_data_model"| timechart span=1m count(volume) by ReportKey as F |
timewrap 1w series=short |
sort -_time |
where _time>=relative_time(now(),"-10m@m") |
eval Time=strftime(_time,"%D %H:%M")|
rename F_s1 as 1_week_ago |
rename F_s0 as Today |
table Time Today 1_weeks_ago
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...