Splunk Search

How to subtract Field values from incremental values

Communicator

This log is updated every 5 minutes (I have included three examples of the logs). The value is cumulative. So, while graphing it in Splunk, I have to deduct the previous value to get the value for that 5 minute interval. I have created 6 fields. So for example lets take one field, pdweb.sescache hit has the following three values of 26965624, 27089514, and 27622280.

Taking 27622280-27089514 = 532766 (this is the actual value I want for that 5 minute interval.

pdweb.sescache hit=
pdweb.sescache miss=
pdweb.sescache add=
pdweb.sescache del=
pdweb.sescache inactive=
pdweb.sescache lifetime=

2015-06-22-11:30:00.000-08:00I----- HPDRA0387I pdweb.sescache statistics report
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache hit : 26965624
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache miss : 1199911
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache add : 767440
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache del : 461681
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache inactive : 307444
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache lifetime : 305720
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache LRU expired : 0

2015-06-22-11:30:00.000-08:00I----- HPDRA0387I pdweb.sescache statistics report
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache hit : 27089514
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache miss : 1187065
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache add : 757928
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache del : 466608
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache inactive : 292919
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache lifetime : 291094
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache LRU expired : 0

2015-06-22-11:30:00.000-08:00I----- HPDRA0387I pdweb.sescache statistics report
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache hit : 27622280
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache miss : 1205104
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache add : 806893
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache del : 491966
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache inactive : 316631
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache lifetime : 314759
2015-06-22-11:30:00.000-08:00I----- pdweb.sescache LRU expired : 0

Tags (2)
1 Solution

Path Finder

Try autoregress: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Autoregress

I cobbled this together from another post [here] that worked for me, but (also works) against multiple hosts.

sourcetype=sescache host=myhost* | sort host, _time
| autoregress sescache_hit as prev_sescache_hit | autoregress host as prev_host
| eval x = (sescache_hit - prev_sescache_hit)
| eval sescache_diff = if(x >= 0, x, null())
| eval sescache_diff = if(host == prev_host, sescache_diff, null())
| timechart span=5m avg(sescache_diff) by host

View solution in original post

Path Finder

Try autoregress: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Autoregress

I cobbled this together from another post [here] that worked for me, but (also works) against multiple hosts.

sourcetype=sescache host=myhost* | sort host, _time
| autoregress sescache_hit as prev_sescache_hit | autoregress host as prev_host
| eval x = (sescache_hit - prev_sescache_hit)
| eval sescache_diff = if(x >= 0, x, null())
| eval sescache_diff = if(host == prev_host, sescache_diff, null())
| timechart span=5m avg(sescache_diff) by host

View solution in original post

Communicator

sourcetype=sescache host=myhost | eval sescache_number = tonumber(sescache_hit) | delta sescache_number p=1

2015-06-23-07:55:00.000-08:00I----- pdweb.sescache hit : 28026932

2015-06-23-07:50:00.000-08:00I----- pdweb.sescache hit : 28020721

With this search Im getting a delta (sescache_number)= -6211

Is there a way to get this in reverse.....I need to subtract the top number (latest event) from the bottom one and the value should be 6211

0 Karma

Communicator

I need to subtract the top number (latest event) from the bottom one and the value should be 6211.

In your example - top number i.e. latest value is 28026932 and bottom one is 28020721.
subtract top number from the bottom one means , 28020721 - 28026932 = -6211 (minus value).

I guess, you want it other way ( subtract bottom number from the top one), to get an positive value?

0 Karma

Communicator

I hope this helps.

sourcetype=sescache host=tstypwsl01 | eval sescache_number = tonumber(sescache_hit) |reverse | table Bytes | delta Bytes p=1

http://answers.splunk.com/answers/46124/graphing-cumulative-counters.html

0 Karma

Communicator

yes looking to get the positive number....

0 Karma

Communicator

sourcetype=sescache | eval sescache_number = tonumber(pdweb.sescache hit) | delta sescache_number p=1

Communicator

If logs are already sorted by _time (Descending order), then try this search and hopefully it should work.

0 Karma

Communicator

Thank you.....you got me to the right path:

sourcetype=sescache host=myhost | eval sescache_hit_diff = tonumber(sescache_hit) | reverse | delta sescache_hit_diff p=1| timechart avg("delta(sescache_hit_diff)")

Im able to chart out the average for the difference between the earliest & latest values!!!

0 Karma

Splunk Employee
Splunk Employee

From the way your question is worded, I'm wondering if it is maybe a multivalued field?

if so...

You could use the mvindex function of the eval command

... | eval delta = tonumber(mvindex('pdweb.sescache hit', -1)) - tonumber(mvindex('pdweb.sescache hit', 0))

This looks for the last value (index at -1) of the multi valued field and subtracts the first (index at 0).
As for doing it for multiple fields, maybe something along the lines of:

...| foreach pdweb.sescache* 
        [ eval delta_<<MATCHSTR>> = tonumber(mvindex(<<FIELD>>, -1)) - tonumber(mvindex(<<FIELD>>, 0)) ]
0 Karma

Communicator

doing this search:

sourcetype=sescache | foreach sescache*
[ eval delta_<> = tonumber(mvindex(<>, -1)) - tonumber(mvindex(<>, 0)) ]

I have the following fields all with the same value 0

delta_hit=
delta_miss=
delta_add=
delta_del=
delta_inactive=
delta_lifetime=

0 Karma

Splunk Employee
Splunk Employee

This wont work because the foreach command requires <> or <> not <>.

0 Karma

Communicator

sourcetype=sescache | eval delta = tonumber(mvindex('sescache_hit', -1)) - tonumber(mvindex('sescache_hit', 0))

Field= sescache_hit
Here are 5-5 min logs
2015-06-22-22:25:00.000-08:00I----- pdweb.sescache hit : 27793603

2015-06-22-22:25:00.000-08:00I----- pdweb.sescache hit : 28313182

2015-06-22-22:25:00.000-08:00I----- pdweb.sescache hit : 27661866

2015-06-22-22:25:00.000-08:00I----- pdweb.sescache hit : 26611428

2015-06-22-22:20:00.000-08:00I----- pdweb.sescache hit : 28309095

this creates the field "delta" which has the only value 0

0 Karma

Splunk Employee
Splunk Employee

All of this was assuming, that it was a multivalued field. If they're all separate events, this wont work. You said you had one field with three values and then all the examples you had in the original question had the exact same timestamp, so I thought that they were all part of the same event ?

0 Karma

Communicator

my bad, each are separate events with a 5 minute interval

2015-06-23-07:20:00.000-08:00I----- pdweb.sescache hit : 27989324

2015-06-23-07:15:00.000-08:00I----- pdweb.sescache hit : 27983667

015-06-23-07:10:00.000-08:00I----- pdweb.sescache hit : 27979265

0 Karma

Splunk Employee
Splunk Employee

So for every event you need the delta from the previous event ? The answer above on delta should work then ?

0 Karma

Communicator

yes...sorry for the confusion......subtract earliest from the latest....example:

lets take the last two events time stamp 7:20 & 7:15
2015-06-23-07:20:00.000-08:00I----- pdweb.sescache hit : 27989324

2015-06-23-07:15:00.000-08:00I----- pdweb.sescache hit : 27983667

so I need to subtract 27989324- 279983667 which would be my true value = 5657....

this comes in as a 5 minute interval....

0 Karma

Communicator

so this is my search:

sourcetype=sescache | eval delta = mvindex('sescache_hit', -1) - mvindex('sescache_hit', 0)

Error in 'eval' command: Typechecking failed. '-' only takes numbers.

Splunk Employee
Splunk Employee

Oops, forgot the tonumber function. Edited my answer.

0 Karma

Communicator

sourcetype=sescache | eval sescache_number = tonumber(pdweb.sescache hit) | delta sescache_number p=1

0 Karma

Builder
0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!