Splunk Search

How to convert large bytes to human readable units (e.g. Kib, MiB, GiB)

tfujita_splunk
Splunk Employee
Splunk Employee

Numeral system macros for Splunk v1.1.1

Bytes to printing Human readable size (e.g. 4KiB, 1023.4MiB, 23.4GiB, 345,67TiB)

Sometimes it is necessary to divide Bytes by powers of 1024 and convert it to human readable units.
In that case, it would not look good if you write that calculation in the SPL each time and the SPL becomes long, so I think it would be good if we could use a common macro to make it simple.

For this purpose, I added 2 macros to Numeral system macros for Splunk v1.1.1.

  • numeral_binary_symbol(bytes)
    • Binary symbol. KiB, MiB, GiB, TiB, PiB, EiB, ZiB, YiB, RiB, QiB
  • numeral_binary_symbol(bytes,digits)
    • Binary symbol with arg for rounding digits.

For other macros provided, click here

Usage1

| makeresults count=35
```THIS SECTION IS JUST CREATING SAMPLE VALUES.```
| streamstats count as digit
| eval val=pow(10,digit-1), val=val+random()%val
| foreach bytes [eval <<FIELD>>=val]
| table digit val bytes
| fieldformat val=tostring(val,"commas")

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 9s",`numeral_binary_symbol(bytes,1)`)
 

tfujita_splunk_1-1675551290092.png

Usage2

Example of sorting sourcetypes in descending order of throughput.

index="_internal" source="*metrics.log" per_sourcetype_thruput
| stats sum(eval(kb*1024)) AS bytes by series

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 10s",`numeral_binary_symbol(bytes,2)`)
| sort 0 - bytes

tfujita_splunk_2-1675551417439.png

Points :  The internal value is still in Byte. Sortable.

  • The kb information can be converted to bytes and a common macro can be used.
  • Since the fieldformat retains the original value internally, the MiB and KiB displays can also be used for sorting, with the values being comparable.

Why weird units KiB, MiB using instead of KB, MB?

As a side note, in the general public, the definition of “kilo” is 1000 and has no other meaning, but in the computer world, it has long been a common belief that KB (Killo Byte) is 1024 bytes to the 10th power of 2, as if it were common knowledge in the industry.

However, this is definitely a source of confusion, so standards such as IEC 60027-2, IEEE 1541-2002, and IEC 80000-13:2008 defines the KiB (Kibibyte) and MiB (Mebibyte) units as byte units based on 1024 bytes to avoid confusion.

These units are not at all widespread and unfamiliar to us, but since confusion over numbers is a source of misunderstanding, I dared to use these units in that macros in order to avoid misunderstanding and to have a common understanding in Splunk’s output.

Enjoy Splunking!

 

 

Labels (5)
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...