Splunk Search

How to convert large bytes to human readable units (e.g. Kib, MiB, GiB)

tfujita_splunk
Splunk Employee
Splunk Employee

Numeral system macros for Splunk v1.1.1

Bytes to printing Human readable size (e.g. 4KiB, 1023.4MiB, 23.4GiB, 345,67TiB)

Sometimes it is necessary to divide Bytes by powers of 1024 and convert it to human readable units.
In that case, it would not look good if you write that calculation in the SPL each time and the SPL becomes long, so I think it would be good if we could use a common macro to make it simple.

For this purpose, I added 2 macros to Numeral system macros for Splunk v1.1.1.

  • numeral_binary_symbol(bytes)
    • Binary symbol. KiB, MiB, GiB, TiB, PiB, EiB, ZiB, YiB, RiB, QiB
  • numeral_binary_symbol(bytes,digits)
    • Binary symbol with arg for rounding digits.

For other macros provided, click here

Usage1

| makeresults count=35
```THIS SECTION IS JUST CREATING SAMPLE VALUES.```
| streamstats count as digit
| eval val=pow(10,digit-1), val=val+random()%val
| foreach bytes [eval <<FIELD>>=val]
| table digit val bytes
| fieldformat val=tostring(val,"commas")

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 9s",`numeral_binary_symbol(bytes,1)`)
 

tfujita_splunk_1-1675551290092.png

Usage2

Example of sorting sourcetypes in descending order of throughput.

index="_internal" source="*metrics.log" per_sourcetype_thruput
| stats sum(eval(kb*1024)) AS bytes by series

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 10s",`numeral_binary_symbol(bytes,2)`)
| sort 0 - bytes

tfujita_splunk_2-1675551417439.png

Points :  The internal value is still in Byte. Sortable.

  • The kb information can be converted to bytes and a common macro can be used.
  • Since the fieldformat retains the original value internally, the MiB and KiB displays can also be used for sorting, with the values being comparable.

Why weird units KiB, MiB using instead of KB, MB?

As a side note, in the general public, the definition of “kilo” is 1000 and has no other meaning, but in the computer world, it has long been a common belief that KB (Killo Byte) is 1024 bytes to the 10th power of 2, as if it were common knowledge in the industry.

However, this is definitely a source of confusion, so standards such as IEC 60027-2, IEEE 1541-2002, and IEC 80000-13:2008 defines the KiB (Kibibyte) and MiB (Mebibyte) units as byte units based on 1024 bytes to avoid confusion.

These units are not at all widespread and unfamiliar to us, but since confusion over numbers is a source of misunderstanding, I dared to use these units in that macros in order to avoid misunderstanding and to have a common understanding in Splunk’s output.

Enjoy Splunking!

 

 

Labels (5)
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...