Dashboards & Visualizations

Field Mapping with Transforms

davecroto
Splunk Employee
Splunk Employee

Can I get an assist for this? My fields are not showing up. Trying a simple extraction / field mapping with transforms.conf.

Here is a sample event:

Thu Oct 31 17:04:56 EDT 2013 bla bla bla (IV_ADM_ID,TOT_ORD_TCK_QTY,AS_ORD_IV_LNAM) VALUE (4,5,6)

I want the fields:

IV_ADM_ID=4
TOT_ORD_TCK_QTY=5
AS_ORD_IV_LNAM=6

transforms.conf:

alt text

props.conf:

alt text

brenttoo is my sourcetype

Tags (1)
0 Karma

Ayn
Legend

First of all, you should NOT be using index-time field extractions (TRANSFORMS) unless you really know what you're doing and why. In this case one reason why your fields are not showing up could have been that you were looking at data that has already been indexed, so this field extraction had not been applied to them. However I see issues with your regex - btw putting it in an image was kind of a bad idea because if someone (like for instance, me) wanted to debug your regex they (I) have to manually write it again.

Anyhow, on to the regex: in your first group you're matching one character A to Z, followed by one or more , characters. Underscores will not be matched at all. So, the text "I,,," would be matched, but for instance "IVADMID," would not. Even less so with "IV_ADM_ID,". There are a couple of other things as well but to make a long story short, this regex works better:

\((\w+),(\w+),(\w+)\)\s+VALUE\s+\((\d+),(\d+),(\d+)\)

It's always a good idea to test this stuff before applying it. I often use http://regexpal.com/ or http://gskinner.com/RegExr/ .

0 Karma

Ayn
Legend

I think you should switch to a REPORT statement first of all to make sure your extraction works. That way you won't have to reindex data every time you want to try out your extraction. After you've got that working, IF you have VERY good reasons for doing this as an index-time transform instead, switch over then.

0 Karma

davecroto
Splunk Employee
Splunk Employee

sorry:

REGEX=\((\w+),(\w+),(\w+)\)\s+VALUE\s+\((\d+),(\d+),(\d+)\)

0 Karma

davecroto
Splunk Employee
Splunk Employee

Thanks Ayn regardless of the multiple regex I've tried and work, (rubular) including yours (real world is a bit more sophisticated) the test data is being cleaned and re=indexed.

I understand your pain about the images of .conf files, Apologies:

props:

[brenttoo]
TRANSFORMS-make_fields_two = make_fields_two

transforms.conf

[make_fields_two]
REGEX=((\w+),(\w+),(\w+))\s+VALUE\s+((\d+),(\d+),(\d+))
FORMAT = $1::$4 $2::$5 $3::$6

Still does not want to work. I would like to get this small example working and then decide if index time is right for this use case.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...