Splunk Search

field extraction not launched at search time

Builder

I've made an extraction (inline) when I test it, in the extraction manager, it works properly.

I have saved it, as field inline extraction, and I thought that each time I would do a search over the source mentioned in the extraction, that the extraction would run automatically, and extract the field, to display them on the left side of the websearch, but that's not the case.

Is it necessary to give a command in the search to extract fields ?

0 Karma
1 Solution

Splunk Employee
Splunk Employee

If you created an EXTRACT rule, and specified a field name that would come from the "automatic" key=value extraction, then it won't work.

For example, if I have a log event like

14 Nov 2012 09:51:22 hostname process[2135]: status=sent:abc123:OK

And my goal is to extract the string "abc123" from the status field, I might write a regex using the colons as a boundary with a rule like this:

EXTRACT-status_subfield = \:(?<remote_id>\w+)\: in status

This pre-supposes that the "status" field has been extracted already. Because EXTRACT rules are applied before the automatic KV extraction that would set status equal to "sent:abc123:OK", there is no "status" field for the regex to work on. If your EXTRACT stanza doesn't specify a source field, it should still be OK to extract the value.

You might have to expressly extract the "status" field first (with another EXTRACT rule) or adjust your regex to find the string you want in the raw data (hidden field called "_raw").

View solution in original post

Builder

I think it works now, I not sure why, but the field are now appearing... I made a couple of changes and I'm not sure which made it, but its ok now !

Many thanks for your help

0 Karma

Champion

you can click the tick on this answer, the tick is located just below the up/down arrows with the 0. If sowings answer helped you to resolve this then you should click the tick on their answer.

Builder

where can I close the question or mark as solved ?

0 Karma

Splunk Employee
Splunk Employee

If you created an EXTRACT rule, and specified a field name that would come from the "automatic" key=value extraction, then it won't work.

For example, if I have a log event like

14 Nov 2012 09:51:22 hostname process[2135]: status=sent:abc123:OK

And my goal is to extract the string "abc123" from the status field, I might write a regex using the colons as a boundary with a rule like this:

EXTRACT-status_subfield = \:(?<remote_id>\w+)\: in status

This pre-supposes that the "status" field has been extracted already. Because EXTRACT rules are applied before the automatic KV extraction that would set status equal to "sent:abc123:OK", there is no "status" field for the regex to work on. If your EXTRACT stanza doesn't specify a source field, it should still be OK to extract the value.

You might have to expressly extract the "status" field first (with another EXTRACT rule) or adjust your regex to find the string you want in the raw data (hidden field called "_raw").

View solution in original post

Splunk Employee
Splunk Employee

If explicitly calling the extraction rule by | extract didn't work, then I suggest that there may be an issue with your regex. You said in your original post that you tested the regex in the extraction manager; have you tried it as the argument to the rex command in the search bar? I wonder whether some of the fixed widths are a character or two off?

Can you update your original question with some sample data?

0 Karma

Builder

I've tried the following at search time :
| EXTRACT vdv_message does not work, results returned but no field extracted

0 Karma

Builder

I think I begining to understand, to extract multiple field at one time I had to make a transform, so I've added a transform name vdv_messages as
.{1}(?P.{23}).{87}(?P.{3}).{5}(?P.{3}).{2}(?P.{3}).{2}(?P.{19})

over field _row

But nothing is happening automatically, who am I supposed to invoke the extraction now ?

0 Karma

Builder

In my case, I'd like to extract serveral field from _raw, as it is a fixed width format, I've tried
.{1}(?P.{23}).{87}(?P.{3}).{5}(?P.{3}).{2}(?P.{3}).{2}(?P.{19})

So I was execting the extraction searching in _raw, and extracting all the fields in the regex

  • Do I have to specify "in _raw" at the end ?
  • or is the problem because I have a star in my source (it is filtered out in the comment here) ?
  • Or a multifield extraction has to be defined somewhere else ?
0 Karma

Champion

After you created your field extraction what is the scope? Are you trying to share it with others, what app did you define it under. Go into Manager>>fields> Field Extraction and click permissions. Allow every one to read and object should apear in all apps. Check if that helps.

0 Karma

Builder

I have no access to the servier itself only the splunk manager. Here is the entry for my extraction :

source::cus-vdv*dfi" OR source="cus-vdv*ans : EXTRACT-vdv_business_messages_head

Inline

.{1}(?P.{23}).{87}(?P.{3}).{5}(?P.{3}).{2}(?P.{3}).{2}(?P.{19})

0 Karma

Splunk Employee
Splunk Employee

The field extraction should only extract one field. Take a look at the props.conf and post it up in your answer as well as your inputs.conf for the data.

0 Karma

Builder

In fact I've used the extractor to correct the regex.

I'm using the same user for the extraction.

I'm using a source="..../name*/" that is working at searchtime but could that be the problem ?

The field extraction, is supposed to extract multiple fields, it that a problem ?

0 Karma

Splunk Employee
Splunk Employee

You used the interactive field extractor to create a field for you? Then yes, if you gave the field a name and saved it, then you should be able to see that extraction automatically for that sourcetype. Are you logged in as the same user? Go to Manager » Fields » Field extractions because each extraction has permissions depending on which users need to see it.

Do you see extraction in /etc/system/local/props.conf?

0 Karma