Splunk Search

File Name Parts Extraction Rex

JDukeSplunk
Builder

I need to break down a source file name into it's meaningful parts with a regex, however the convention of the file changes a little depending on what type of log it is.

I've pulled the field "file" from source with

| rex field=source "(?P<file>[^\\\]+)$"

And this tables out to something like these.

Over all for dev, for that specific day

*HealthDataFax-DEV-2019-03-29.csv*

Over all for UAT for that month.

*HealthDataFax-UAT-2019-03.csv*

And here's where my brain breaks..
Environment data for a specific user and specific yyyy-mm-dd.

*HealthDataFax-PROD-Username-2019-03-29.csv*

So I'd like to be able to turn each of these file pieces into a field without having to tell the developer he needs to reformat these file names.

FileName/Environment/Username/Year/Month/Day

Thanks,

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try (runanywhere sample, everything before rex is to generate sample data)

| gentimes start=-1 | eval file="Health-DEV-2019-03-29.csv HEALTH-UAT-2019-03.csv HEALTH-PROD-user1-2019-03-29.csv" | table file | makemv file | mvexpand file 
| rex field=file "^(?<FileName>[^-]+)-(?<Env>[^-]+)-(?<user>[^-]*)-*(?<Year>\d{4})-(?<Month>\d{2})-*(?<Date>\d*)\.csv"

View solution in original post

0 Karma

somesoni2
Revered Legend

Give this a try (runanywhere sample, everything before rex is to generate sample data)

| gentimes start=-1 | eval file="Health-DEV-2019-03-29.csv HEALTH-UAT-2019-03.csv HEALTH-PROD-user1-2019-03-29.csv" | table file | makemv file | mvexpand file 
| rex field=file "^(?<FileName>[^-]+)-(?<Env>[^-]+)-(?<user>[^-]*)-*(?<Year>\d{4})-(?<Month>\d{2})-*(?<Date>\d*)\.csv"
0 Karma

vnravikumar
Champion

Hi

Give a try

| makeresults 
| eval file="HealthDataFax-PROD-Username-2019-03-29.csv" 
| eval temp = split(file,"-"), FileName= mvindex(temp,0),Environment=mvindex(temp,1),Username=mvindex(temp,2),Year=mvindex(temp,3),Month=mvindex(temp,4),Date=mvindex(split(mvindex(temp,5),"."),0) |table FileName,Environment,Username,Year,Month,Date

OR

| makeresults 
| eval file="HealthDataFax-PROD-Username-2019-03-29.csv" 
| rex field=file "(?P<FileName>[^-]+)-(?P<Environment>[^-]+)-(?P<Username>[^-]+)-(?P<Year>[^-]+)-(?P<Month>[^-]+)-(?P<Day>[^.]+)"
0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...