Splunk Search

FileSize to human readable

tb5821
Communicator

I'm surprised splunk doesn't have an easier way to get a human readable format by passing it the field you want it to work its magic on al-la du -h

So here's my question, I have a field called fs which is file size but its in bytes. I want to convert that to the proper Kb,Mb,Gb format... whats the best way to do this?

Tags (4)
0 Karma
1 Solution

alacercogitatus
SplunkTrust
SplunkTrust

I have this setup. You can put it into $SPLUNK_HOME/etc/system/local/props.conf and it should work. You may want to have your field called "b" or "bytes" for it to work, or change the config to match your fields. This will auto-calculate kilos, megas and gigas automatically.

[host::*]
priority = 100
EVAL-kilobytes = if(isnotnull(kilobytes),kilobytes,bytes/1024)
EVAL-megabytes = if(isnotnull(megabytes),megabytes,bytes/1024/1024)
EVAL-gigabytes = if(isnotnull(gigabytes),gigabytes,bytes/1024/1024/1024)
EVAL-kb = if(isnotnull(kb),kb,b/1024)
EVAL-mb = if(isnotnull(mb),mb,b/1024/1024)
EVAL-gb = if(isnotnull(gb),gb,b/1024/1024/1024)

You can also use a macro (in the Search UI):
Manager -> Advanced Search -> Search Macros

Name: resize(1)
Arguments: bytes
definition = eval kilobytes = $bytes$/1024| eval megabytes=kilobytes/1024|eval gigabytes/1024

Then you can do your_search | resize(fs)

View solution in original post

jameswatts
Explorer

How about just installing Humanize?

Convert numbers, bytes, and timestamps into fuzzy, human-friendly units! Using the humanize library from https://github.com/jmoiron/humanize

Here are some example searches:
index=_internal | head | eval foo=random() | humanize command=naturalsize foo
index=_internal earliest=-7h | bin _time span=10m | eval foo=_time | stats first(foo) as foo by _time | humanize command=naturaltime foo

https://splunkbase.splunk.com/app/3104/

0 Karma

jpvlsmv
Path Finder

Ancient thread necropsy, but here's a better macro (IMO). It's ugly but it works just like the -h option on many GNU tools.
Usage:

| eval readable_size=`readable(size)`

Definition: (as seen in Settings -> Adv Search -> Search macros -> new:

if( $num$ < 1024, tostring($num$), if ( (floor($num$/pow(1024,floor(log($num$,1024))))) < 10
     , ( (tostring((floor($num$/pow(1024,floor(log($num$,1024)))))) + ".") + tostring(round((($num$/pow(1024,floor(log($num$,1024))))-(floor($num$/pow(1024,floor(log($num$,1024))))))*10))) + (substr("KMGTPEZY",floor(log($num$,1024)),1))
     , ( tostring((floor($num$/pow(1024,floor(log($num$,1024)))))) + (substr("KMGTPEZY",floor(log($num$,1024)),1)) )
   ) )

Not an eval-based definition (unchecked)
Arguments: num
Validation Expression: !isnum($num$)
Validation Error Message: Numeric value required

My key observation for the algorithm is that the log base 1024 will give you the "scale"-- KB or PB or whatever, by dropping the fractional part (i.e. log_10(5.6MB) = 2 -> M).

In working on this, I used meaningful names and replace-all'd them to fundamental eval functions. Here's the pseudocode:

if $num$ < 1024:
  printf("%4d", $num$)
else
  if $num$ reduces to a single digit
    # print in the form x.yS
    printf( "%d.%d%c", whole_part(reduction), 1st digit of frac_part(reduction), KMGTPEZY suffix appropriate for this scale
  else # This is actually the most common case.  The result is just the whole part of the reduction and the suffix
    printf("%3d%s", whole_part(reduction), suffix)

Hope this helps somebody
--Joe

alacercogitatus
SplunkTrust
SplunkTrust

I have this setup. You can put it into $SPLUNK_HOME/etc/system/local/props.conf and it should work. You may want to have your field called "b" or "bytes" for it to work, or change the config to match your fields. This will auto-calculate kilos, megas and gigas automatically.

[host::*]
priority = 100
EVAL-kilobytes = if(isnotnull(kilobytes),kilobytes,bytes/1024)
EVAL-megabytes = if(isnotnull(megabytes),megabytes,bytes/1024/1024)
EVAL-gigabytes = if(isnotnull(gigabytes),gigabytes,bytes/1024/1024/1024)
EVAL-kb = if(isnotnull(kb),kb,b/1024)
EVAL-mb = if(isnotnull(mb),mb,b/1024/1024)
EVAL-gb = if(isnotnull(gb),gb,b/1024/1024/1024)

You can also use a macro (in the Search UI):
Manager -> Advanced Search -> Search Macros

Name: resize(1)
Arguments: bytes
definition = eval kilobytes = $bytes$/1024| eval megabytes=kilobytes/1024|eval gigabytes/1024

Then you can do your_search | resize(fs)

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

weird. anyway, glad that worked for you!

0 Karma

tb5821
Communicator

Found the issue, the definition needs to be:

eval kilobytes=($fs$/1024) | eval megabytes=kilobytes/1024 |eval gigabytes=megabytes/1024

0 Karma

tb5821
Communicator

now getting:
Error in 'eval' command: Failed to parse the provided arguments. Usage: eval dest_key = expression

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

you need the "backticks". They are the key under Esc on the keyboard.

0 Karma

tb5821
Communicator

Error in 'SearchParser': Missing a search command before '''.

...| 'resize(fs)'

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

See my edit. You will need backticks around "resize(fs)". The answer system stripped them for me.

0 Karma

tb5821
Communicator

Can I do it in search? I don't have access to anything but the search UI

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!