topic Re: Get difference of occurence between 2 different day/months/years/date ranges in Splunk Search

Get difference of occurence between 2 different day/months/years/date ranges

isvnplunk — Thu, 05 Aug 2021 07:40:43 GMT

Hi all,

First post here - So I'm a Splunk beginner & recently got this tricky task.

So let's say I have these rows in my log file:

2020-01-01: error778
2020-01-02: error778
2020-01-03: error778
2020-01-16: error778
2020-02-01: error778
2020-02-04: error778
2020-02-06: error778
2020-02-10: error778
2020-02-18: error778
2020-02-19: error778

On Jan 2020, we can see that there are 4 rows of error778
On Feb 2020, we can see that there are 6 rows of error778
This means, from Jan 2020 to Feb 2020, there's 50% diff/increase of error778.

The questions:

How can I get/display the % difference? Ideally, the delimiters can be days, month, year, or date ranges (such as, diff of error778 between 1-5 Jan 2020 and 5-31 Jan 2020).
What's the best way to set an alert based on % (say, alert when diff is > 15%)?

I'm able to display the daily/weekly/monthly trend of a keyword using timechart like below

index=mylog "error778" | timechart span=1month count by date

But I believe it's far from what I need.

Any help would be appreciated, thanks.

Re: Get difference of occurence between 2 different day/months/years/date ranges

ITWhisperer — Thu, 05 Aug 2021 07:47:49 GMT

Try something like this

Re: Get difference of occurence between 2 different day/months/years/date ranges

isvnplunk — Thu, 05 Aug 2021 07:59:04 GMT

Much appreciated, @ITWhisperer , it is very close to what I need.

I assume if it needs to be yearly or weekly or daily, I just need to switch the bin _time span from 1mon to 1year/1day etc?

Would it very difficult to make it between some date ranges instead?

Also, now that we have the "percent" field, how do I create an alert based on this? Say, notify us if the percent of error is > 15%, for example.

Re: Get difference of occurence between 2 different day/months/years/date ranges

ITWhisperer — Thu, 05 Aug 2021 09:06:59 GMT

Yes, essentially what bin is doing is setting the field, _time in this instance, to the beginning of the period, e.g. month, week, day, hour, minute etc. Then the stats are gathered for that value. If you want different periods, then you could evaluate a field, e.g. _time based on whether it is before or after say 5th of month and set it to one value or another.

For alerts, just add a where command so that results are only returned when the threshold has been breached and set your alarm based on whether results exist

Re: Get difference of occurence between 2 different day/months/years/date ranges

isvnplunk — Thu, 05 Aug 2021 09:11:24 GMT

Noted, will do as suggested, thank you @ITWhisperer !