Splunk Search

How to apply a field extractor created to a search ?

rcrisan09
Engager

I created a field extractor for different fields for an event. Now I would like to search all the events from a source and apply that field extractor to see the fields that I'm interested in. The field extractor seems to appear, but I don't know how to apply that into my search. I've tried REPORT- but no luck. How can I apply a field extractor already created into a search ?

Labels (2)

css1971
Engager

Original question was posed in 2017.

Now, in 2024, 7 years later it is still not very clear how one applies a saved extraction regex to an existing search to extract fields from the search. Especially without access to the various server side configuration files. Splunk has grown long in the tooth, dementia encroaching.

Reality: You probably can't do it simply.

If you have a sourcetype X. The extractors you saved will only run against the base, plain data set sent as X, not against your search, and they run against the base sourcetype automatically. If it was going to work, it would already be working and you would already have your field.

Now, if your search does any kind of transformations like for example pulling log fields out of JSON data using spath, messing around with _raw or similar, the extractor you created isn't going to run against that resulting data set. I know, I've tried. The extractors get applied before that part of the search runs.

See: https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Knowledge/Searchtimeoperationssequence

You're going to have to go into Settings -> Fields -> Field Extractions and copy/paste the regex created by the web extractor page into your search and manually extract the field within your search using the "rex" command. You may have to tweak it slightly especially for quotes.

It's a little disingenuous of the splunk web extraction generator to take the results of the current search as the input and imply that a saved extractor will actually operate against such a search and pull fields out for you. It doesn't.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Use the extract command. For example, if you have a field extractor in a stanza in transforms.conf called "foo" then you would use it this way.

<your base search> | extract foo | ...
---
If this reply helps you, Karma would be appreciated.

otheus
Explorer

Not a useful answer. The question concerned a field extractor, not a transform. Are you implying that the ONLY way Splunk can use a field-extractor is to first create a transform? Pity, since that seems beyond the scope of an ordinary user.

0 Karma

t_danen
New Member

it cannot be done. Splunk is stupid and non-intuitive or maybe they want to sell professional services like ITRS does,it cannot be done the way you want it. you have to plunk down the regex in its entirety.
Splunk is stupid and non-intuitive or maybe they want to sell professional services like ITRS does

0 Karma

jkat54
SplunkTrust
SplunkTrust

Except they do sell PS and stay busy helping people who won't read the manual or for whatever reason can't find the time to.

Each field extraction is applied to a sourcetype generally. The extractions are only going to work on the sourcetypes they've been setup for, and only in the apps they've been configured in (unless the extraction is set to global sharing), and only for those users who have read permission on the app they are found in.

It's complicated, but only if you don't take the time to study the material and your environment first.

romanwaldecker
Path Finder

I have the same issue here. And I cannot access the transforms.conf file (or the server's file system at all) to get the Stanza of my field extractor.
In the Splunk Web-UI in the field extractions overview, the name of my field extractor is like my_sourcetype : EXTRACT-my_new_field.
Is there any other way to derive the Stanza through the Splunk Web-UI?

MuS
Legend

Hi romanwaldecker,

Late to the party, but yes this can be done using the UI. But you need to understand the differences in the various possible field extractions that can be configured in props.conf.

  • EXTRACT is an inline search time regex field extraction that is not linked to transforms.conf
  • REPORT is a search time field extraction that is linked to transforms.conf
  • TRANSFORMS is a index-time/parsing field extraction

since you have an EXTRACT option configured there is no transforms.conf stanza linked.
An example for a REPORT option is the default field extraction of splunk_web_access which you can see using this URI:

 http[s]://YourSplunkServer:YourPort/en-GB/manager/launcher/data/props/extractions/splunk_web_access%20%3A%20REPORT-access?action=edit&ns=system&f_sort_key=value&f_sort_dir=asc&uri=%2FservicesNS%2Fnobody%2Fsystem%2Fdata%2Fprops%2Fextractions%2Fsplunk_web_access%2520%253A%2520REPORT-access

The transform stanza name will be access-extractions which in turn could be used with the extract command like this:

<your base search> | extract access-extractions | ...

Hope this helps ...

cheers, MuS

otheus
Explorer

Your answer from 2020 was very unclear, less clear than the documentation. OK, so here goes: Splunk provides a fascinating way to search and report on log data, and promises simplicity in various use-cases. One (would think) extremely common use-case is for users in the enterprise edition to create custom regular expressions in order to extract values from select log lines, and then do various things with those extracted values. 

The documentation and GUI lead one to think one can create a python-perl extended regex to extract such fields. However, instead of then being able to _use_ such a regex, the user must _save_ it somehow with a name. And then the documentation goes off in the weeds without any explanation as to how to _use_ such saved extractions.

There's lots of discussion about props.conf and transforms.conf, but this appears to predate the enterprise edition, in which ordinary users do not have such godlike powers over a centralized, enteprise splunk deployment.

So as simply as possible, please tell me what additional steps an ordinary user within an Splunk enterprise deploymnet must take in order to create searches and then later reports and alerts using saved field-extractions.

0 Karma

Marcos_Vilas
Engager

Hello Mus, i think i'm the latest guy ever to this party.

 

I don't think i got the point here, so if we create our field extraction (regular expression) trough the UI, it would be a EXTRACTION option configured right ? 

I have created my field in the same way that romanwaldecker did, and got the same name for my extraction:

my_sourcetype : EXTRACT-MYFIELD

but, when I try to do a search 

<your base search> | extract MYFIELD 

 it keeps getting me these error:

Error in 'extract' command: Failed to parse the key-value pair configuration for transform 'MYFIELD'.

Do you possibly have in mind what it could be ? I'm kinda trapped on it for a few days

shivanshu1593
Builder

Hello @MuS,

I'm even more late to the party, but am running in somewhat of a similar situation. I have new data coming in via syslog, but no fields are auto extracted. So, I'm using REPORT to extract them. I have the stanza ready, but I placed it in the Heavy forwarder by mistake. Should I place it in the props on the search head or the Indexer for the change to work.

Thank you,

Thank you,
Shiv
###If you found the answer helpful, kindly consider upvoting/accepting it as the answer as it helps other Splunkers find the solutions to similar issues###

MuS
Legend

Hi there, since you're using REPORT it has to go on the Search Head like written, explained above:

  • REPORT is a search time field extraction that is linked to transforms.conf

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...