Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Splunk Search

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Community
- :
- Splunk Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- How to get stats command to calculate with precisi...

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

dmr195

Communicator

08-11-2014
09:45 AM

I have found that the stats command's output doesn't use scientific notation. This means that if I need to calculate some statistics on a set of very small numbers then stats just reports 0.00000.

You can see the effect with this search:

```
| stats count | eval small="1.23456e-306" | stats min(small)
```

It will always return 0.00000.

Is there a way to have the stats command in the above search keep the precision and return 1.23456e-306?

(I know in this example the stats command is pointless because there's only one value, but the real case is that I have many events containing very small numbers and I want to calculate various statistics of the data set without all the small numbers getting changed to 0.)

1 Solution

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

somesoni2

Revered Legend

08-11-2014
10:38 AM

Try something like this

```
| stats count | eval small="1.23456e-30" | eval small=small*1 | stats min(small)
```

PS: The example value that you gave little too small so changed to little big number.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

dmr195

Communicator

08-12-2014
02:37 AM

A possibility for getting around the loss of precision in stats is to take logarithms before the stats command and then convert back afterwards. This works quite well:

```
| stats count | eval small="1.23456e-304" | eval lnsmall=ln(small) | stats min(lnsmall) as minlnsmall | eval minsmall=if(minlnsmall<-702.0, 0, exp(minlnsmall*1.0))
```

The check for minlnsmall being less than -702 is needed to avoid nan being returned close to the limits of double precision floating point accuracy, and it's necessary to multiply minlnsmall by 1.0 rather than 1 otherwise some premature rounding occurs sometimes (but not always).

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

somesoni2

Revered Legend

08-11-2014
10:38 AM

Try something like this

```
| stats count | eval small="1.23456e-30" | eval small=small*1 | stats min(small)
```

PS: The example value that you gave little too small so changed to little big number.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

dmr195

Communicator

08-12-2014
02:06 AM

Another interesting thing is that eval can handle the full double precision accuracy, for example:

| stats count | eval small="1.23456e-303" | eval small=small*1

prints small with 308 digits to the right of the decimal point.

So it seems like it's the stats command that's limited to 125 decimal places.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

dmr195

Communicator

08-12-2014
01:38 AM

It seems like 125 digits to the right of the decimal point is the most that's possible. So:

| stats count | eval small="1.23456e-125" | eval small=small*1 | stats min(small)

goes to:

0.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001

and:

| stats count | eval small="1.23456e-126" | eval small=small*1 | stats min(small)

loses all precision and prints 0.

This is a shame as the smallest double precision floating point number is 2.2e-308. But with I'm better off than I was originally, so thanks for the answer!

Register for .conf21 Now!
Go Vegas or Go Virtual!