topic Re: override field with transforms.conf in Getting Data In

How to override field with transforms.conf?

rafadvega — Fri, 26 May 2023 12:22:14 GMT

I have a sourcetype with events like:

fieldname.field1=value1,fieldname.field2=value1 value2 value3 value4,fieldname.field3=value1

To extract the fields, I use the following transformation which works correctly:

[extract_key_value] FORMAT = $1::$2 REGEX = fieldname.([^=]+)=([^\,]+)

The field2, I want to convert it into a multivalue field by splitting it on spaces. If I do the following, it works correctly:

[field2] SOURCE_KEY = field2 MV_ADD = 1 FORMAT = field_test::$1 REGEX = ([^\s]+):

However, if I try to overwrite the field, it doesn't work.

[field2] SOURCE_KEY = field2 MV_ADD = 1 FORMAT = field2::$1 REGEX = ([^\s]+):

What could be my mistake?
Thank you in advance

PD:

My props.conf:

[mysourcetype] KV_MODE = None REPORT-extract_key_value = extract_key_value REPORT-extract_mv_fields = field2 SHOULD_LINEMERGE = false

Re: override field with transforms.conf

rafadvega — Fri, 26 May 2023 10:35:27 GMT

I have found an alternative way using FIELDALIAS. If there is a better way to do it, I appreciate the help:

props.conf:

[mysourcetype] FIELDALIAS-field2_mv_as_field2= field2_mv AS field2 KV_MODE = None REPORT-extract_key_value = extract_key_value REPORT-extract_mv_fields = field2 SHOULD_LINEMERGE = false

transforms.conf

[extract_key_value] FORMAT = $1::$2 REGEX = fieldname.([^=]+)=([^\,]+) [field2] SOURCE_KEY = field2 MV_ADD = 1 FORMAT = field2_mv::$1 REGEX = ([^\s]+):

Re: override field with transforms.conf

isoutamo — Fri, 26 May 2023 11:16:53 GMT

This is quite interesting case ,-) Obviously at least docs needs information that you cannot use fieldX as target. You should leave note on docs about this.

Another way to do this is follow The sequence of search-time operations which shows the order of applied actions and select how you can get work around for this.

Another way to do this is use calculated field.

Add this to your sourcetype definition. This can do via conf file or via GUI

[mysourcetype] REPORT-extract_key_value = extract_key_value EVAL-field2 = split(field2, " ")

and keep your transformation as earlier

[extract_key_value] FORMAT = $1::$2 REGEX = fieldname.([^=]+)=([^\,]+)

This way there shouldn't any additional fields on your field list.

r. Ismo