Splunk Search

Field Extraction from existing field

sloshburch
Splunk Employee
Splunk Employee

Although this works with no issue in SPL:

 | rex field=fieldName "(?i)^(?P<test>.*)$"

This

EXTRACT-test = (?i)^(?P<test>.*)$ in fieldname

seems to ONLY work when fieldname is source, sourcetype, host, etc.. - but does not work when fieldname is any of the fields that splunk auto-discovers within the events (name=value pairs).

Running Splunk 6.0.2. I could swear this worked in prior releases.

http://docs.splunk.com/Documentation/Splunk/6.0.2/Admin/Propsconf
shows that the syntax would be:

EXTRACT-<class> = [<regex>|<regex> in <src_field>]

And provides instructions:

  • Use ' in ' to match the regex against the values of a specific field. Otherwise it just matches against _raw (all raw event data).
  • NOTE: can only contain alphanumeric characters (a-z, A-Z, and 0-9).
  • If your regex needs to end with 'in ' where is not a field name, change the regex to end with '[i]n ' to ensure that Splunk doesn't try to match to a field name.

But my "fieldname" is only alpha characters and yet it still does not work.

I did not see anything listed on the Known Issues page for 6.0.2 regarding field extractions.

Any ideas?

0 Karma
1 Solution

rdownie
Communicator

I had a similar issue and it turned out to be the order in which the extractions occurred.

View solution in original post

0 Karma

rdownie
Communicator

I had a similar issue and it turned out to be the order in which the extractions occurred.

0 Karma

BP9906
Builder

So whats the solution to determine the order of extraction?

0 Karma

fortiwhall
Explorer

I'm having same problem. Source logs have a key=value pair called "ui"
ui=GUI(x.x.x.x)
ui=ssh(x.x.x.x)
ui=console
ui=https
etc

I want to make a CIM-compliant field called 'app' for Authentication since it's supposed to specify the mechanism. But I want to only catch the field value up until the first parenthesis.

This line in my local props.conf works (working against entire _raw field)

EXTRACT-fgt_auth_app_extract = ui=['"]?(?[^(\s'"]+)['"\s]?

but this line does not (trying to use the "in ui" specifier at the end)

EXTRACT-fgt_auth_app_extract = ['"]?(?[^(\s'"]+)['"\s]? in ui
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I think auto-extraction happens after props.conf/transforms.conf extraction, so those fields aren't available yet.

lakshman239
SplunkTrust
SplunkTrust

In the case of delimited files (e.g IIS/w3c, tab delimited files), the field extraction happens at index time right? So, these fields are available as part of name value pairs in the search time. So [regex | regex in ] doesn't work. It seems accept only source. Is this a bug?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...