Splunk Search

Index time extraction: How to field extract from source, with two capturing groups?

splunkIT
Splunk Employee
Splunk Employee

This is an example of my source:

/frameworks/app_console-ui_v656_web_0/runs/latest/errors.stdout

I am using the following props and transforms for my test case:

## props.conf
[mytest]
TRANSFORMS-extract-source = extract-source
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false

## transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

I have confirmed that the regex used is correct, but it's not working if "SOURCE_KEY = MetaData:Source" is used.

1 Solution

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

View solution in original post

ifeldshteyn
Communicator

Thank you, I was going nuts trying to figure out where source:: was coming from.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...