Splunk Search
Highlighted

Cumulative counters with "slice" logic

Path Finder

Hi all!

I am working on task: Create cumulative chart for counting Success and Error entities, by 1 hour slice interval, with checking latest [Status] value by [ID] and [StatusDateTime] to every [Slice].

"Slice logic" - for example, exist next Events:

ID   Status   StatusDateTime
------------------------------
1    Error    2014-04-23 10:55
2    Success  2014-04-23 10:55
1    Success  2014-04-23 11:55

Need to get next result:

Slice              Success    Error
------------------------------------
2014-04-23 11:00   1          1
2014-04-23 12:00   2          0

I know how to calculate count separately for 1 hour periods:

index="log_index"  
| eval GroupDate=strftime(relative_time(StatusDateTime, "+1h@h"), "%Y-%m-%d %H:%M")  
| stats latest(Status) as Status by ID, GroupDate  
| stats c(eval(Status="Success")) as SuccessCount, c(eval(Status="Error")) as ErrorCount by GroupDate

In SQL, I can do subqueries for each period and calculate it (specifying latest in Sub-Search as GroupDate). But, as I understood, Splunk does not support passing parameters/values from Main-Search to Sub-Search, is it true?

I do not have any ideas how to create needed cumulative logic.
Anyone can guide me please on this?

Thanks!

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

SplunkTrust
SplunkTrust

Something like this?

index="log_index" | bucket span=1h _time as slice | dedup ID slice | timechart span=1h count by Status

The bucket will take care of your one-hour-slices, and the dedup will discard all but the latest event per slice for every ID.

Highlighted

Re: Cumulative counters with "slice" logic

Path Finder

Martin, great thanks!

But, it is not cumulative. With this search, I'll get in slice 2014-04-23 12:00 - 1 Success and 0 Error.

I know about "accum" and "delta" operators, but they doesn't allow realize cumulative "slice logic by ID" fully.

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

SplunkTrust
SplunkTrust

Append this after the timechart:

... | accum Error | accum Success

How does that not calculate your cumulative values?

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

Path Finder

Martin, it's not fully what I need. I described task in the my first post. As you can see, in the second slice total Success increased, but total Error decreased.

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

SplunkTrust
SplunkTrust

In my mind a cumulative value cannot decrease, so maybe we're thinking of different things.

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

Path Finder

Therefore I called it as "slice cumulative logic" 🙂
I understand that it's unordinary logic, but there is such requirements.

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

SplunkTrust
SplunkTrust

Googling that term yields this question as the top result for me: https://www.google.com/search?q=slice+cumulative+logic
I guess you'll have to explain what you mean by that...

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

Path Finder

Hm... Sorry, I'm not sure that it's realy called as "slice cumulative logic", it's only my version. Therefore I tried to describe it in the first post.

Explanation: At every slice, search must include all events in previous slices. For example: if earliest boundary it's 8:00 AM, then:
1) At 9:00 AM - Includes events from 8:00 to 9:00 AM;
2) At 10:00 AM - Includes events from 8:00 to 10:00 AM;
3) At 11:00 AM - Includes events from 8:00 to 11:00 AM;
and so on.

0 Karma
Highlighted

Re: Cumulative counters with "slice" logic

SplunkTrust
SplunkTrust

Isn't that what accum does?

0 Karma