Splunk Search

Index time extraction: How to field extract from source, with two capturing groups?

splunkIT
Splunk Employee
Splunk Employee

This is an example of my source:

/frameworks/app_console-ui_v656_web_0/runs/latest/errors.stdout

I am using the following props and transforms for my test case:

## props.conf
[mytest]
TRANSFORMS-extract-source = extract-source
DATETIME_CONFIG = CURRENT
NO_BINARY_CHECK = true
category = Custom
pulldown_type = true
SHOULD_LINEMERGE = false
disabled = false

## transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

I have confirmed that the regex used is correct, but it's not working if "SOURCE_KEY = MetaData:Source" is used.

1 Solution

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

View solution in original post

ifeldshteyn
Path Finder

Thank you, I was going nuts trying to figure out where source:: was coming from.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

Looks like if an anchor is used at the beginning of the regex, it needs to include source:: as this is how it's written in the bucket's Sources.data metadata file.

This revised stanza appears to be working for me now:

##  transforms.conf 
[extract-source]
SOURCE_KEY = MetaData:Source 
REGEX = ^source::\/frameworks\/app_console-ui_v([\d]+)_([^_]+)_
FORMAT = job_id::$2.$1 
WRITE_META = true

Hope this information will help someone encountering similar issues.

View solution in original post

Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!