Splunk Search

Converting variable File Size Units with dot points to bytes

satiex
Explorer

Hi there,

I am building a Synology Splunk TA to share with the community. In the logs, file sizes can be presented in many different units:

 

1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB

 

 

I've been stuck trying to convert all of these values to bytes.

This post was really helpful in using regex and eval statements, but does not consider the added complexity of have decimal places.

Any assistance is appreciated and will be credited in the App.

Labels (5)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB" 
| multikv noheader=t 
| rex "^(?<Value>[\d.]+)\s?(?<Unit>\w*)$" 
| eval factor=case(Unit="B",1,Unit="KB",1024,Unit="MB",1024*1024,Unit="GB",1024*1024*1024,Unit="TB",11024*1024*1024*1024,true(),1) 
| eval InBytes=Value*factor
| table _raw InBytes

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults
| eval _raw="1.72 KB
2.35 KB
0 Bytes
75.08 KB
243.00 KB
18.62 MB
261.62 KB
48.60 GB" 
| multikv noheader=t 
| rex "^(?<Value>[\d.]+)\s?(?<Unit>\w*)$" 
| eval factor=case(Unit="B",1,Unit="KB",1024,Unit="MB",1024*1024,Unit="GB",1024*1024*1024,Unit="TB",11024*1024*1024*1024,true(),1) 
| eval InBytes=Value*factor
| table _raw InBytes

satiex
Explorer

Thanks for the quick response - using this website the conversions are accurate. 

Here is a link to the app in development: https://github.com/satiex/splunk_synology_TA 

Does anyone know how I can get these conversions to happen at index time? I've tried a few things with the props.conf and transforms.conf without any luck.

 

Here is an example of a log event:

Sep 19 13:19:54 172.17.0.1 Sep 19 13:19:54 SYN-NAS WinFileService Event: read, Path: /Shows/TV Show/Season 1/Episode 1 - Pilot.mkv, File/Folder: File, Size: 1.23 GB, User: john.smith@ldap.local.com, IP: 192.168.1.59

And here is the REGEX in the transforms.conf:

[synology_ft_basefields]
REGEX = \sWinFileService Event:\s(?<action>create|write|read|delete|rename),\sPath:\s(?<path>.+?(?=,)),\sFile\/Folder:\s(?<file_folder>File|Folder),\sSize:\s(?<sizeValue>[\d.]+)\s?(?<sizeUnit>\w*),\sUser:\s(?<user>[^\,]*),\sIP:\s(?<src_ip>[[ipv4]])

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...