Splunk Search

How to convert large bytes to human readable units (e.g. Kib, MiB, GiB)

tfujita_splunk
Splunk Employee
Splunk Employee

Numeral system macros for Splunk v1.1.1

Bytes to printing Human readable size (e.g. 4KiB, 1023.4MiB, 23.4GiB, 345,67TiB)

Sometimes it is necessary to divide Bytes by powers of 1024 and convert it to human readable units.
In that case, it would not look good if you write that calculation in the SPL each time and the SPL becomes long, so I think it would be good if we could use a common macro to make it simple.

For this purpose, I added 2 macros to Numeral system macros for Splunk v1.1.1.

  • numeral_binary_symbol(bytes)
    • Binary symbol. KiB, MiB, GiB, TiB, PiB, EiB, ZiB, YiB, RiB, QiB
  • numeral_binary_symbol(bytes,digits)
    • Binary symbol with arg for rounding digits.

For other macros provided, click here

Usage1

| makeresults count=35
```THIS SECTION IS JUST CREATING SAMPLE VALUES.```
| streamstats count as digit
| eval val=pow(10,digit-1), val=val+random()%val
| foreach bytes [eval <<FIELD>>=val]
| table digit val bytes
| fieldformat val=tostring(val,"commas")

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 9s",`numeral_binary_symbol(bytes,1)`)
 

tfujita_splunk_1-1675551290092.png

Usage2

Example of sorting sourcetypes in descending order of throughput.

index="_internal" source="*metrics.log" per_sourcetype_thruput
| stats sum(eval(kb*1024)) AS bytes by series

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 10s",`numeral_binary_symbol(bytes,2)`)
| sort 0 - bytes

tfujita_splunk_2-1675551417439.png

Points :  The internal value is still in Byte. Sortable.

  • The kb information can be converted to bytes and a common macro can be used.
  • Since the fieldformat retains the original value internally, the MiB and KiB displays can also be used for sorting, with the values being comparable.

Why weird units KiB, MiB using instead of KB, MB?

As a side note, in the general public, the definition of “kilo” is 1000 and has no other meaning, but in the computer world, it has long been a common belief that KB (Killo Byte) is 1024 bytes to the 10th power of 2, as if it were common knowledge in the industry.

However, this is definitely a source of confusion, so standards such as IEC 60027-2, IEEE 1541-2002, and IEC 80000-13:2008 defines the KiB (Kibibyte) and MiB (Mebibyte) units as byte units based on 1024 bytes to avoid confusion.

These units are not at all widespread and unfamiliar to us, but since confusion over numbers is a source of misunderstanding, I dared to use these units in that macros in order to avoid misunderstanding and to have a common understanding in Splunk’s output.

Enjoy Splunking!

 

 

Labels (4)
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Agent Mode Engaged! Enchaining Agentic Operations with Splunk AI Assistant 2.0

    Are you ready to transform how your team handles complex data requests? We invite you to our upcoming ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...