topic Re: FileSize to human readable in Splunk Search

Does Splunk have an easier way to get FileSize to human readable format?

tb5821 — Sun, 05 Feb 2023 23:17:14 GMT

I'm surprised splunk doesn't have an easier way to get a human readable format by passing it the field you want it to work its magic on al-la du -h

So here's my question, I have a field called fs which is file size but its in bytes. I want to convert that to the proper Kb,Mb,Gb format... whats the best way to do this?

Re: FileSize to human readable

alacercogitatus — Thu, 25 Jul 2013 15:38:39 GMT

I have this setup. You can put it into $SPLUNK_HOME/etc/system/local/props.conf and it should work. You may want to have your field called "b" or "bytes" for it to work, or change the config to match your fields. This will auto-calculate kilos, megas and gigas automatically.

[host::*] priority = 100 EVAL-kilobytes = if(isnotnull(kilobytes),kilobytes,bytes/1024) EVAL-megabytes = if(isnotnull(megabytes),megabytes,bytes/1024/1024) EVAL-gigabytes = if(isnotnull(gigabytes),gigabytes,bytes/1024/1024/1024) EVAL-kb = if(isnotnull(kb),kb,b/1024) EVAL-mb = if(isnotnull(mb),mb,b/1024/1024) EVAL-gb = if(isnotnull(gb),gb,b/1024/1024/1024)

You can also use a macro (in the Search UI):
Manager -> Advanced Search -> Search Macros

Name: resize(1) Arguments: bytes definition = eval kilobytes = $bytes$/1024| eval megabytes=kilobytes/1024|eval gigabytes/1024

Then you can do your_search | resize(fs)

Re: FileSize to human readable

tb5821 — Thu, 25 Jul 2013 15:43:45 GMT

Can I do it in search? I don't have access to anything but the search UI

Re: FileSize to human readable

alacercogitatus — Thu, 25 Jul 2013 15:49:06 GMT

See my edit. You will need backticks around "resize(fs)". The answer system stripped them for me.

Re: FileSize to human readable

tb5821 — Thu, 25 Jul 2013 15:57:17 GMT

Error in 'SearchParser': Missing a search command before '''.

...| 'resize(fs)'

Re: FileSize to human readable

alacercogitatus — Thu, 25 Jul 2013 15:58:55 GMT

you need the "backticks". They are the key under Esc on the keyboard.

Re: FileSize to human readable

tb5821 — Thu, 25 Jul 2013 16:23:28 GMT

now getting:
Error in 'eval' command: Failed to parse the provided arguments. Usage: eval dest_key = expression

Re: FileSize to human readable

tb5821 — Thu, 25 Jul 2013 16:58:45 GMT

Found the issue, the definition needs to be:

eval kilobytes=($fs$/1024) | eval megabytes=kilobytes/1024 |eval gigabytes=megabytes/1024

Re: FileSize to human readable

alacercogitatus — Thu, 25 Jul 2013 17:04:59 GMT

weird. anyway, glad that worked for you!

Re: FileSize to human readable

jpvlsmv — Fri, 21 Jul 2017 14:48:19 GMT

Ancient thread necropsy, but here's a better macro (IMO). It's ugly but it works just like the -h option on many GNU tools.
Usage:

| eval readable_size=`readable(size)`

Definition: (as seen in Settings -> Adv Search -> Search macros -> new:

if( $num$ < 1024, tostring($num$), if ( (floor($num$/pow(1024,floor(log($num$,1024))))) < 10
     , ( (tostring((floor($num$/pow(1024,floor(log($num$,1024)))))) + ".") + tostring(round((($num$/pow(1024,floor(log($num$,1024))))-(floor($num$/pow(1024,floor(log($num$,1024))))))*10))) + (substr("KMGTPEZY",floor(log($num$,1024)),1))
     , ( tostring((floor($num$/pow(1024,floor(log($num$,1024)))))) + (substr("KMGTPEZY",floor(log($num$,1024)),1)) )
   ) )

Not an eval-based definition (unchecked)
Arguments: num
Validation Expression: !isnum($num$)
Validation Error Message: Numeric value required

My key observation for the algorithm is that the log base 1024 will give you the "scale"-- KB or PB or whatever, by dropping the fractional part (i.e. log_10(5.6MB) = 2 -> M).

In working on this, I used meaningful names and replace-all'd them to fundamental eval functions. Here's the pseudocode:

if $num$ < 1024:
  printf("%4d", $num$)
else
  if $num$ reduces to a single digit
    # print in the form x.yS
    printf( "%d.%d%c", whole_part(reduction), 1st digit of frac_part(reduction), KMGTPEZY suffix appropriate for this scale
  else # This is actually the most common case.  The result is just the whole part of the reduction and the suffix
    printf("%3d%s", whole_part(reduction), suffix)

Hope this helps somebody
--Joe

Re: FileSize to human readable

jameswatts — Tue, 29 Sep 2020 15:00:52 GMT

How about just installing Humanize?

Convert numbers, bytes, and timestamps into fuzzy, human-friendly units! Using the humanize library from https://github.com/jmoiron/humanize

https://splunkbase.splunk.com/app/3104/

Re: FileSize to human readable

tfujita_splunk — Sat, 04 Feb 2023 12:35:10 GMT

This could be solution for you.

https://community.splunk.com/t5/Dashboards-Visualizations/Smart-conversion-of-large-numbers-to-human-readable-for-use-on/m-p/629593/highlight/true#M51640

| makeresults count=35
```THIS SECTION IS JUST CREATING SAMPLE VALUES.```
| streamstats count as digit
| eval val=pow(10,digit-1), val=val+random()%val
| foreach bytes [eval <<FIELD>>=val]
| table digit val bytes
| fieldformat val=tostring(val,"commas")

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 10s",printf("%.2f",round(bytes/pow(1024,if(bytes=0,0,floor(min(log(bytes,1024),10)))),2)).case(bytes=0 OR log(bytes,1024)<1,"B ", log(bytes,1024)<2,"KiB", log(bytes,1024)<3,"MiB", log(bytes,1024)<4,"GiB", log(bytes,1024)<5,"TiB", log(bytes,1024)<6,"PiB", log(bytes,1024)<7,"EiB", log(bytes,1024)<8,"ZiB", log(bytes,1024)<9,"YiB", log(bytes,1024)<10,"RiB", log(bytes,1024)<11,"QiB", 1=1, "QiB"))

If you can install app or ask admin on your to install app,

installing add-on Numeral system macros for Splunk enables you to use macros numeral_binary_symbol(1) or numeral_binary_symbol(2).

Example

| makeresults count=35
```THIS SECTION IS JUST CREATING SAMPLE VALUES.```
| streamstats count as digit
| eval val=pow(10,digit-1), val=val+random()%val
| foreach bytes [eval <<FIELD>>=val]
| table digit val bytes
| fieldformat val=tostring(val,"commas")

```THE FOLLOWING LINES MAY BE WHAT ACHIEVES THE FORMAT YOU ARE LOOKING FOR.```
| fieldformat bytes=printf("% 10s",`numeral_binary_symbol(bytes,2)`)

Numeral system macros for Splunk

https://splunkbase.splunk.com/app/6595

Usage:

How to convert a large number to string with expressions of long and short scales, or neither.

https://community.splunk.com/t5/Splunk-Search/How-to-convert-a-large-number-to-string-with-expressions-of-long/m-p/629383