topic How to count events in a time frame based on a time elapsed field in Splunk Search

How to count events in a time frame based on a time elapsed field

CarbonCriterium — Wed, 08 Jun 2022 13:58:06 GMT

What is the is the best approach to creating a field that shows the number of incomplete requests in a given period of time?

For the machine in question, events are logged when it completes the Request-Response Loop.
I have a field `time_taken` which shows, in milliseconds, how long the Request-Response Loop has taken.
I have already done the following, now how do I evaluate the total number of `open_requests` for each second?

Re: How to count events in a time frame based on a time elapsed field

yuanliu — Wed, 08 Jun 2022 16:07:47 GMT

It looks like the challenge is how to define the requirement, i.e., the difference between _time at the beginning of the pseudo code which you use as a marker of "responded", and _time at the end of the pseudo code which you intend as a marker of clock unit (second).

I assume that fields _time and time_taken, therefore responded and requested as well, are all in time format, i.e., can be used in numeric comparisons. Ignoring the strftime() calculations which are meant for display only, the following can give you something meaningful:

| eval responded = _time | eval requested = _time - time_taken | bin _time span=1s ``` chop _time into 1-s bins ``` | where requested < _time AND time_taken > 1s ``` many ways to construct this, depending on interpretation and preference ``` | timechart count

Hope this helps.

Re: How to count events in a time frame based on a time elapsed field

CarbonCriterium — Wed, 08 Jun 2022 21:44:20 GMT

Thanks, I eventually came to something similar! I think this is the solution I am after, unless you can spot a hole in the logic.

| eval seconds_taken = time_taken/1000 | eval responded = _time, requested = _time - seconds_taken | where requested <= responded AND seconds_taken > 0 ``` | where requested <= responded AND seconds_taken >= 0 ``` | timechart count span=1s

Re: How to count events in a time frame based on a time elapsed field

yuanliu — Thu, 09 Jun 2022 11:51:59 GMT

As long as you test a variety of data manually and are satisfied with the results, there should be no concern.

This said, both conditions "requested <= responded" and "seconds_taken > 0" will always be true. Shouldn't it be "seconds_taken > 1"? ("requested <= responded" is always true.) At the bottom of this, any event in which time_taken > 1000 would be characterized as "open request" because you wanted to count from the end of each second.

To get the results logically sound, you also want to shift time axis according to requested, something like

On the other hand, now that I look it from this angle, there's another consideration that needs attention: If an event's seconds_taken > 2 but < 3, the event should be counted as "open request" in two 1s bins; the "open" state will be concurrent with other "open" requests (older and newer) for the entire duration. Effectively, you would be stacking Gantt charts.

I faced a very similar problem years ago that somesoni2 helped solve. You can see the answer in https://community.splunk.com/t5/Splunk-Search/How-to-compute-concurrent-members-in-events/m-p/112163#M29438