topic Re: Some byte values not converting to GB, while others are. in All Apps and Add-ons

Some byte values not converting to GB, while others are.

HCadmins — Sat, 07 Jan 2017 17:48:24 GMT

Here is a report I run monthly to get each virtual machine's provisioned, committed, and uncommitted storage, using the excellent VMware app.

sourcetype=vmware:inv:vm 'HandleFourHourWindowEndConditional' host="<our vcenter hostname>" | spath moid output=vmmoid | spath changeSet.name output=vmname | spath changeSet.layoutEx.file{}.name output=filelist | spath changeSet.layoutEx.file{}.size output=filesizes | spath changeSet.layoutEx.file{}.type output=filetypes | spath changeSet.storage.perDatastoreUsage{}.committed output=committed | spath changeSet.storage.perDatastoreUsage{}.uncommitted output=uncommitted | spath changeSet.summary.runtime.host.moid output=vmhost | search NOT (<an esxi host we want to exclude> OR <another esxi host we want to exclude) | dedup vmname, vmmoid, host | eval provisioned=committed+uncommitted | rex field=filelist "\[(?<datastore>.*)\] .+/(?<filenames>[^/]+)" | eval dsname=mvindex(datastore,0) | search dsname=* host="<our vcenter server>" | eval uncommitted='format_bytes(uncommitted)' | eval committed='format_bytes(committed)' | eval provisioned='format_bytes(provisioned)' | rename vmhost as HIDE-HostSystem-moid, vmmoid as HIDE-VirtualMachine-moid, vmname as "VirtualMachine", filenames as "Individual Files", filesizes as "Individual File Sizes", filetypes as "File Types", uncommitted as "Uncommitted Space", committed as "Committed Space", provisioned as "Provisioned Space" | table _time HIDE-HostSystem-moid host HIDE-VirtualMachine-moid "HostSystem" "VirtualMachine" "Provisioned Space" "Committed Space" "Uncommitted Space" "Individual Files" "Individual File Sizes" "File Types" | lookup FullHierarchy host, moid AS HIDE-HostSystem-moid, host AS host OUTPUT name AS HostSystem | fields - _time, HIDE*, Ind*, host, File* | sort +VirtualMachine

This report runs fine, but the calculations for storage space, which should be in GB, output in bytes on some of the machines. See screenshot:

I wonder if there's a way to say

IF "Provisioned Space" = "Null Bytes" THEN convert Committed Space and Uncommitted Space to GB

Re: Some byte values not converting to GB, while others are.

niketn — Tue, 29 Sep 2020 12:19:39 GMT

Can you confirm whether committed and uncommitted fields are numeric prior to being fed to format_bytes or not? If format_bytes appends GB and Bytes to the end of numeric value then the It would be better to handle in format_bytes macro, if not you can use if() condition to handle GB and Bytes. Otherwise, you would need to share the format_bytes macro code from Splunk Knowledge Object.

Re: Some byte values not converting to GB, while others are.

HCadmins — Sun, 08 Jan 2017 03:54:47 GMT

Here's the code for the format_bytes macro

if($bytes$>1073741824, tostring(round($bytes$/1073741824,2))+" GB", if($bytes$>1048576, tostring(round($bytes$/1048576,2))+" MB", if($bytes$>1024, tostring(round($bytes$/1024))+" KB", tostring($bytes$)+" Bytes")))

Re: Some byte values not converting to GB, while others are.

HCadmins — Sun, 08 Jan 2017 03:56:09 GMT

I'm not sure how to confirm whether the committed and uncommitted fields are numeric prior to being fed to format_bytes, but I would assume they are since it's taking them from vCenter (which I am only assuming).

Re: Some byte values not converting to GB, while others are.

niketn — Sun, 08 Jan 2017 05:58:10 GMT

Your macro is getting inputs as bytes, however, you existing default case treats any value greater than 1 GB as bytes, which is incorrect. Try the following case statements instead of nested if. I have account for Tera Byte and Zeta Byte on top of your existing case. Anything greater that 1024 TB will show up as ZB through final case statement.

eval $bytes$=case($bytes$>=1125899906842624 AND $bytes$>1152921504606846976, tostring(round($bytes$/1152921504606846976,2))+" ZB",$bytes$>=1099511627776 AND committed<1125899906842624,tostring(round($bytes$/1073741824,2))+" TB",$bytes$>=1073741824 AND $bytes$<1099511627776, tostring(round($bytes$/1073741824,2))+" GB", $bytes$>=1048576 AND $bytes$<1073741824, tostring(round($bytes$/1048576,2))+" MB", $bytes$>1024 AND $bytes$<1073741824, tostring(round($bytes$/1024,2))+" KB", $bytes$<1024,tostring($bytes$+" Bytes"),1=1,tostring(round($bytes$/1152921504606846976,2))+" ZB")

PS: Based on the same you have provided, where your if statement starts my example will start with case statement. So replace macro definition accordingly.

Re: Some byte values not converting to GB, while others are.

HCadmins — Tue, 10 Jan 2017 22:39:14 GMT

Thank you niketnilay!

Would you say this is a bug in the VMware app?

Re: Some byte values not converting to GB, while others are.

niketn — Tue, 11 Jul 2017 16:56:53 GMT

If the above macro is defined in VMWare App and not created by you, then this is a bug in VMWare app, as evidently it is not handling >1GB

Re: Some byte values not converting to GB, while others are.

DUThibault — Thu, 25 Jan 2018 18:16:58 GMT

The format_bytes code yields gibibytes (GiB), not gigabytes (GB). Use either: if($bytes$>1073741824, tostring(round($bytes$/1073741824,2))+" GiB", if($bytes$>1048576, tostring(round($bytes$/1048576,2))+" MiB", if($bytes$>1024, tostring(round($bytes$/1024))+" KiB", tostring($bytes$)+" Bytes"))) or if($bytes$>1000000000, tostring(round($bytes$/1000000000,2))+" GB", if($bytes$>1000000, tostring(round($bytes$/1000000,2))+" MB", if($bytes$>1000, tostring(round($bytes$/1000))+" kB", tostring($bytes$)+" Bytes")))

Re: Some byte values not converting to GB, while others are.

DUThibault — Thu, 25 Jan 2018 18:33:41 GMT

The niketnilay code is incorrect. A zebibyte is not 1152921504606846976 bytes (that's an exbibyte) but rather 1180591620717411303424 bytes. This code should be closer:
eval $bytes$=case($bytes$>=1152921504606846976, tostring(round($bytes$/1152921504606846976,2))+" EiB",$bytes$>=1125899906842624 AND $bytes$<1152921504606846976,tostring(round($bytes$/1125899906842624,2))+" PiB",$bytes$>=1099511627776 AND $bytes$<1125899906842624,tostring(round($bytes$/1099511627776,2))+" TiB",$bytes$>=1073741824 AND $bytes$<1099511627776, tostring(round($bytes$/1073741824,2))+" GiB", $bytes$>=1048576 AND $bytes$<1073741824, tostring(round($bytes$/1048576,2))+" MiB", $bytes$>=1024 AND $bytes$<1048576, tostring(round($bytes$/1024,2))+" KiB", $bytes$<1024,tostring($bytes$+" Bytes"),1=1,tostring(round($bytes$/1180591620717411303424,2))+" ZiB")