topic Is there any documentation on what precision Splunk uses for arithmetic operations? in Splunk Search

Is there any documentation on what precision Splunk uses for arithmetic operations?

krdo — Tue, 10 Nov 2015 13:50:32 GMT

When I execute the following search

index="does not matter"
| stats count AS value
| eval value=123456.0
| eval x=value/(1024*1024*1024)
| eval y=(value*1.0)/(1024*1024*1024)

I get results which do not quite meet my expectations:

value: 123456.0
x: 0.0001149774
y: 0.00011

Seems like splunk truncates the value. But why? Multiplying by 1.0 should (could, might, ...) transform the value into a floating point number but strangely this results in a less precise result. Is there any documentation on which precision splunk uses for arithmetic operations?

Re: Is there any documentation on what precision Splunk uses for arithmetic operations?

HeinzWaescher — Tue, 10 Nov 2015 13:55:19 GMT

I think this helps:

https://answers.splunk.com/answers/320265/why-is-splunk-sometimes-rounding-to-the-next-integ.html

Re: Is there any documentation on what precision Splunk uses for arithmetic operations?

krdo — Tue, 10 Nov 2015 14:14:50 GMT

It seems like the documentation explaining how splunk handles precision has been removed; http://docs.splunk.com/Documentation/Splunk/4.2.5/SearchReference/Eval is the last version containing the section Significant figures and precision which is pretty important IMHO.