topic Re: Delta for different fields? in Splunk Search

Delta for different fields?

nbharadwaj — Tue, 11 May 2010 07:08:49 GMT

I need to calculate the object allocation rate in the jvm. That value can be derived from the jvmlogs. If I subtract the "heapBeforeGC" field in one line from the “heapAfterGC” field in the previous line and divide it by the elapse time I can get a good estimate for how many bytes we allocate in the jvm per unit of time.
I need to trend that value.

Delta provides a diff between the same field in adjacent events. What can I use to diff different fields in adjacent events, and then trend that?

Here are couple of log lines:

2010-05-11T00:09:26.079+0000: 325107.330: [GC [PSYoungGen: 451418K->42291K(453312K)] 1847697K->1438569K(1874624K), 0.0203820 secs] [Times: user=0.25 sys=0.01, real=0.02 secs]

2010-05-11T00:09:29.776+0000: 325111.027: [GC [PSYoungGen: 453299K->45942K(460864K)] 1849577K->1442220K(1882176K), 0.0228840 secs] [Times: user=0.27 sys=0.01, real=0.03 secs]

I want to diff the fields in bold for successive events

Re: Delta for different fields?

gkanapathy — Tue, 11 May 2010 11:06:57 GMT

Please look at the delta search command: http://www.splunk.com/base/Documentation/latest/SearchReference/Delta

Re: Delta for different fields?

Lowell — Wed, 12 May 2010 03:40:12 GMT

You might also find eventstats useful here, if your looking for some data trending over a few events.

Re: Delta for different fields?

nbharadwaj — Wed, 12 May 2010 03:41:35 GMT

Delta only works for the same field. I am trying to subtract fieldA in current event from fieldB on previous event

Re: Delta for different fields?

gkanapathy — Wed, 12 May 2010 04:45:33 GMT

You can rename or simply eval fields to a field with the same name before applying delta.

Re: Delta for different fields?

nbharadwaj — Thu, 13 May 2010 10:56:11 GMT

:A=1:B=2
:A=10:B=20
My delta needs to be: B from time2 minus A from time1. If I eval both fields to the same variable, how do I differentiate? Can u give me an example please?

Re: Delta for different fields?

sideview — Fri, 14 May 2010 09:50:45 GMT

UPDATE: What you want to do, is use streamstats and eval, as in the following.

| streamstats current=f first(heapAfterGC) as previousheapAfterGC | eval difference=heapBeforeGC-previousheapAfterGC

Neither delta nor autoregress will be of much use to you.

Delta makes a lot of sense, except that it seems it cant handle this case where the two fields are different. You could maybe eval the old field to the new field and somehow get a multivalued field then that has both values, but im not sure what to do then...

But It looks like the autoregress command might give you a different way:
http://www.splunk.com/base/Documentation/latest/SearchReference/Autoregress

<your search> | autoregress heapAfterGC AS previousHeapAfterGC p=1 | eval delta=heapAfterGC-previousHeapAfterGC

Autoregress was created primarily to give you moving averages (with p>1), but it seems to work fine with p=1 too.

Re: Delta for different fields?

nbharadwaj — Fri, 14 May 2010 12:32:09 GMT

Thanks that might help. However I am using 4.0.10, and it seems autoregress is available in 4.1 and above. Need to upgrade!