Field Extraction from existing field

[sloshburch]

Although this works with no issue in SPL:

 | rex field=fieldName "(?i)^(?P<test>.*)$"

This

EXTRACT-test = (?i)^(?P<test>.*)$ in fieldname

seems to ONLY work when fieldname is source, sourcetype, host, etc.. - but does not work when fieldname is any of the fields that splunk auto-discovers within the events (name=value pairs).

Running Splunk 6.0.2. I could swear this worked in prior releases.

http://docs.splunk.com/Documentation/Splunk/6.0.2/Admin/Propsconf
shows that the syntax would be:

EXTRACT-<class> = [<regex>|<regex> in <src_field>]

And provides instructions:

• Use ' in ' to match the regex against the values of a specific field. Otherwise it just matches against _raw (all raw event data).
• NOTE: can only contain alphanumeric characters (a-z, A-Z, and 0-9).
• If your regex needs to end with 'in ' where is not a field name, change the regex to end with '[i]n ' to ensure that Splunk doesn't try to match to a field name.

But my "fieldname" is only alpha characters and yet it still does not work.

I did not see anything listed on the Known Issues page for 6.0.2 regarding field extractions.

Any ideas?

[rdownie · Accepted solution]

I had a similar issue and it turned out to be the order in which the extractions occurred.

[BP9906]

So whats the solution to determine the order of extraction?

[fortiwhall]

I'm having same problem. Source logs have a key=value pair called "ui"
ui=GUI(x.x.x.x)
ui=ssh(x.x.x.x)
ui=console
ui=https
etc

I want to make a CIM-compliant field called 'app' for Authentication since it's supposed to specify the mechanism. But I want to only catch the field value up until the first parenthesis.

This line in my local props.conf works (working against entire _raw field)

EXTRACT-fgt_auth_app_extract = ui=['"]?(?[^(\s'"]+)['"\s]?

but this line does not (trying to use the "in ui" specifier at the end)

EXTRACT-fgt_auth_app_extract = ['"]?(?[^(\s'"]+)['"\s]? in ui

[martin_mueller]

I think auto-extraction happens after props.conf/transforms.conf extraction, so those fields aren't available yet.

[lakshman239]

In the case of delimited files (e.g IIS/w3c, tab delimited files), the field extraction happens at index time right? So, these fields are available as part of name value pairs in the search time. So [regex | regex in ] doesn't work. It seems accept only source. Is this a bug?