Splunk Search

Converting variable File Size Units with dot points to bytes

satiex
Explorer

Hi there,

I am building a Synology Splunk TA to share with the community. In the logs, file sizes can be presented in many different units:

 

1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB

 

 

I've been stuck trying to convert all of these values to bytes.

This post was really helpful in using regex and eval statements, but does not consider the added complexity of have decimal places.

Any assistance is appreciated and will be credited in the App.

Labels (5)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB" 
| multikv noheader=t 
| rex "^(?<Value>[\d.]+)\s?(?<Unit>\w*)$" 
| eval factor=case(Unit="B",1,Unit="KB",1024,Unit="MB",1024*1024,Unit="GB",1024*1024*1024,Unit="TB",11024*1024*1024*1024,true(),1) 
| eval InBytes=Value*factor
| table _raw InBytes

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB" 
| multikv noheader=t 
| rex "^(?<Value>[\d.]+)\s?(?<Unit>\w*)$" 
| eval factor=case(Unit="B",1,Unit="KB",1024,Unit="MB",1024*1024,Unit="GB",1024*1024*1024,Unit="TB",11024*1024*1024*1024,true(),1) 
| eval InBytes=Value*factor
| table _raw InBytes

satiex
Explorer

Thanks for the quick response - using this website the conversions are accurate. 

Here is a link to the app in development: https://github.com/satiex/splunk_synology_TA 

Does anyone know how I can get these conversions to happen at index time? I've tried a few things with the props.conf and transforms.conf without any luck.

 

Here is an example of a log event:

Sep 19 13:19:54 172.17.0.1 Sep 19 13:19:54 SYN-NAS WinFileService Event: read, Path: /Shows/TV Show/Season 1/Episode 1 - Pilot.mkv, File/Folder: File, Size: 1.23 GB, User: john.smith@ldap.local.com, IP: 192.168.1.59

And here is the REGEX in the transforms.conf:

[synology_ft_basefields]
REGEX = \sWinFileService Event:\s(?<action>create|write|read|delete|rename),\sPath:\s(?<path>.+?(?=,)),\sFile\/Folder:\s(?<file_folder>File|Folder),\sSize:\s(?<sizeValue>[\d.]+)\s?(?<sizeUnit>\w*),\sUser:\s(?<user>[^\,]*),\sIP:\s(?<src_ip>[[ipv4]])

 

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...