topic Re: Calculate delta for corresponding fields base on time grouped events in Splunk Search

Calculate delta for corresponding fields base on time grouped events

noveix — Mon, 28 Sep 2020 15:57:39 GMT

Need ideas on how to do field calculations based on 2 sets of transactions. Data file is as follows :

Timestamp_1 field_1 field_2 field_3 field_4 id_1

Timestamp_1 field_1 field_2 field_3 field_4 id_2

Timestamp_1 field_1 field_2 field_3 field_4 id_3

Timestamp_1 field_1 field_2 field_3 field_4 id_4

Timestamp_2 field_1 field_2 field_3 field_4 id_1

Timestamp_2 field_1 field_2 field_3 field_4 id_2

Timestamp_2 field_1 field_2 field_3 field_4 id_3

Timestamp_2 field_1 field_2 field_3 field_4 id_4

Timestamp_3 field_1 field_2 field_3 field_4 id_1

Timestamp_3 field_1 field_2 field_3 field_4 id_2

Timestamp_3 field_1 field_2 field_3 field_4 id_3

Timestamp_3 field_1 field_2 field_3 field_4 id_4

Multiple events have same timestamp (Timestamp_1, Timestamp_2...) at regular intervals. Field_1 .. to Field_4 are cumulative from previous identical timestamped fields.

First part:

Need to calculate DELTA by comparing previous event for the same id_1, id_2 ..etc
Eg. need to calculate Delta as follows :

Timestamp2 Field_1 - Timestamp_1 field_1

Timestamp3 Field_1 - Timestamp_2 field_1

Second part:

Add the DELTAs for field_1, field_2... per time interval, so

Timestamp_2 field-delta_1 field-delta_2 field-delta_3 field_4 id_1

Timestamp_2 field-delta_1 field-delta_2 field-delta_3 field_4 id_2

Timestamp_2 field-delta_1 field-delta_2 field-delta_3 field_4 id_3

Timestamp_2 field-delta_1 field-delta_2 field-delta_3 field_4 id_4

Sum of field-delta_1 for all id_?s

Appreciate any help/pointers with this !!

Re: Calculate delta for corresponding fields base on time grouped events

martin_mueller — Mon, 24 Feb 2014 10:38:59 GMT

For calculating deltas split by a field you can use streamstats:

... | streamstats current=f window=1 last(field) as last_field by split_field | eval delta_field = field - last_field

Not sure what you mean by the second part, could you elaborate?

Re: Calculate delta for corresponding fields base on time grouped events

noveix — Mon, 24 Feb 2014 11:11:13 GMT

Thanks Martin, that works, for the second part .. I need to sum the deltas for all of the id_# for the same timestamp .. using my example data above, deltas for the 4 events which has identical timestamp.

Re: Calculate delta for corresponding fields base on time grouped events

martin_mueller — Mon, 24 Feb 2014 11:53:36 GMT

Do you want to sum up several fields in one event or sum up one field over several events - or both?

Re: Calculate delta for corresponding fields base on time grouped events

noveix — Mon, 28 Sep 2020 15:58:08 GMT

I want to sum up delta of field_1 for all events that have the same timestamp regardless of the id_#

Re: Calculate delta for corresponding fields base on time grouped events

martin_mueller — Mon, 24 Feb 2014 22:44:35 GMT

Ah. Append this:

... | eventstats sum(delta_field_1) by _time

Re: Calculate delta for corresponding fields base on time grouped events

noveix — Mon, 24 Feb 2014 23:12:14 GMT

great work ... works nicely !!