Field Mapping with Transforms

[davecroto]

Can I get an assist for this? My fields are not showing up. Trying a simple extraction / field mapping with transforms.conf.

Here is a sample event:

Thu Oct 31 17:04:56 EDT 2013 bla bla bla (IV_ADM_ID,TOT_ORD_TCK_QTY,AS_ORD_IV_LNAM) VALUE (4,5,6)

I want the fields:

IV_ADM_ID=4
TOT_ORD_TCK_QTY=5
AS_ORD_IV_LNAM=6

transforms.conf:

props.conf:

brenttoo is my sourcetype

[Ayn]

First of all, you should NOT be using index-time field extractions (TRANSFORMS) unless you really know what you're doing and why. In this case one reason why your fields are not showing up could have been that you were looking at data that has already been indexed, so this field extraction had not been applied to them. However I see issues with your regex - btw putting it in an image was kind of a bad idea because if someone (like for instance, me) wanted to debug your regex they (I) have to manually write it again.

Anyhow, on to the regex: in your first group you're matching one character A to Z, followed by one or more , characters. Underscores will not be matched at all. So, the text "I,,," would be matched, but for instance "IVADMID," would not. Even less so with "IV_ADM_ID,". There are a couple of other things as well but to make a long story short, this regex works better:

\((\w+),(\w+),(\w+)\)\s+VALUE\s+\((\d+),(\d+),(\d+)\)

It's always a good idea to test this stuff before applying it. I often use http://regexpal.com/ or http://gskinner.com/RegExr/ .

[Ayn]

I think you should switch to a REPORT statement first of all to make sure your extraction works. That way you won't have to reindex data every time you want to try out your extraction. After you've got that working, IF you have VERY good reasons for doing this as an index-time transform instead, switch over then.

[davecroto]

sorry:

REGEX=\((\w+),(\w+),(\w+)\)\s+VALUE\s+\((\d+),(\d+),(\d+)\)

[davecroto]

Thanks Ayn regardless of the multiple regex I've tried and work, (rubular) including yours (real world is a bit more sophisticated) the test data is being cleaned and re=indexed.

I understand your pain about the images of .conf files, Apologies:

props:

[brenttoo]
TRANSFORMS-make_fields_two = make_fields_two

transforms.conf

[make_fields_two]
REGEX=((\w+),(\w+),(\w+))\s+VALUE\s+((\d+),(\d+),(\d+))
FORMAT = $1::$4 $2::$5 $3::$6

Still does not want to work. I would like to get this small example working and then decide if index time is right for this use case.