topic Re: Field Extraction from existing field in Splunk Search

Field Extraction from existing field

sloshburch — Wed, 16 Apr 2014 16:04:16 GMT

Although this works with no issue in SPL:

 | rex field=fieldName "(?i)^(?P<test>.*)$"

This

EXTRACT-test = (?i)^(?P<test>.*)$ in fieldname

seems to ONLY work when fieldname is source, sourcetype, host, etc.. - but does not work when fieldname is any of the fields that splunk auto-discovers within the events (name=value pairs).

Running Splunk 6.0.2. I could swear this worked in prior releases.

http://docs.splunk.com/Documentation/Splunk/6.0.2/Admin/Propsconf
shows that the syntax would be:

EXTRACT-<class> = [<regex>|<regex> in <src_field>]

And provides instructions:

Use ' in ' to match the regex against the values of a specific field. Otherwise it just matches against _raw (all raw event data).
NOTE: can only contain alphanumeric characters (a-z, A-Z, and 0-9).
If your regex needs to end with 'in ' where is not a field name, change the regex to end with '[i]n ' to ensure that Splunk doesn't try to match to a field name.

But my "fieldname" is only alpha characters and yet it still does not work.

I did not see anything listed on the Known Issues page for 6.0.2 regarding field extractions.

Any ideas?

Re: Field Extraction from existing field

martin_mueller — Wed, 16 Apr 2014 17:26:53 GMT

I think auto-extraction happens after props.conf/transforms.conf extraction, so those fields aren't available yet.

Re: Field Extraction from existing field

rdownie — Wed, 16 Apr 2014 19:02:52 GMT

I had a similar issue and it turned out to be the order in which the extractions occurred.

Re: Field Extraction from existing field

BP9906 — Fri, 10 Jul 2015 15:59:01 GMT

So whats the solution to determine the order of extraction?

Re: Field Extraction from existing field

fortiwhall — Sun, 10 Apr 2016 18:48:11 GMT

I'm having same problem. Source logs have a key=value pair called "ui"
ui=GUI(x.x.x.x)
ui=ssh(x.x.x.x)
ui=console
ui=https
etc

I want to make a CIM-compliant field called 'app' for Authentication since it's supposed to specify the mechanism. But I want to only catch the field value up until the first parenthesis.

This line in my local props.conf works (working against entire _raw field)

EXTRACT-fgt_auth_app_extract = ui=['"]?(?[^(\s'"]+)['"\s]?

but this line does not (trying to use the "in ui" specifier at the end)

EXTRACT-fgt_auth_app_extract = ['"]?(?[^(\s'"]+)['"\s]? in ui

Re: Field Extraction from existing field

lakshman239 — Fri, 31 Mar 2017 14:36:17 GMT

In the case of delimited files (e.g IIS/w3c, tab delimited files), the field extraction happens at index time right? So, these fields are available as part of name value pairs in the search time. So [regex | regex in ] doesn't work. It seems accept only source. Is this a bug?