Splunk Search

Index time extraction: How to field extract from source, with two capturing groups?

splunkIT
Splunk Employee
Splunk Employee

This is an example of my source:

/frameworks/app_console-ui_v656_web_0/runs/latest/errors.stdout

I am using the following props and transforms for my test case:

## props.conf
[mytest]
TRANSFORMS-extract-source = extract-source
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false

## transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

I have confirmed that the regex used is correct, but it's not working if "SOURCE_KEY = MetaData:Source" is used.

1 Solution

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

View solution in original post

ifeldshteyn
Communicator

Thank you, I was going nuts trying to figure out where source:: was coming from.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

Get Updates on the Splunk Community!

Set Up More Secure Configurations in Splunk Enterprise With Config Assist

This blog post is part 3 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...