Splunk 4.3 Field Extraction Tool

[peasead]

I am using the Field Extraction tool that is built in Splunk 4.3 and I am having some issues.

I know that fields are generated at search time and I know that I need to make sure the permissions are set properly.

All that said, here is my issue:

I have created new field extractions, but I am not seeing them in the available fields. My assumption is that the search I am running is "saved" (in the aspect of being cached or something) so the fields aren't being reloaded.

How can I "force" the fields to be reloaded?

[peasead · Accepted solution]

Thanks for all your responses.

I know that 4.3 is supposed to rebuild the fields when you run a new search, however, it appears that until I run a new search (same sourcetype, must have completely new search strings) Splunk won't add the new fields - after they are populated once, I can see them all the time.

I must not be explaining myself properly since I'm getting requests for sample logs and regex examples, I already said that the regex is correct.

Again, I appreciate the attempts.

[Drainy]

We only ask for regex as from experience, a lot of people in the past say they have correct regex and then the fault ends up being with the said regex. Anyway, the behaviour still doesn't sound quite correct. You may be experiencing an issue where Splunk is only extracting them as needed, this can cause fields to show with no results if it hasn't extracted them at search time. Have a look at fields.conf as you can define fields as not being index-time extractions which will force Splunk to extract them before using search terms. Glad you've got it working!

[Drainy]

Splunk will use them from the moment you create them via the field extractor. If you wanted to post your regex and some example log data we could verify that the matches will work.

Something else to be sure about is that if you use the field extractor then the field extractions will be created within the context of the app you are currently in. Lets say you are in a cisco app, you create some field extractions and then switch to the search app. It is entirely possible that due to permissions they are not shared across to the search app (they could also pop up within your own user folder too).

A good check to do is to run this command from SPLUNK_HOME/bin;
./splunk cmd btool <config> list --debug

Where is props or transforms, this will list all field extractions with the app that is applying them before it. Mikes suggestion to paste your regex directly into splunk with rex is a great idea and will test it straight off.

[mikelanghorst]

The only saved searching it should be doing is in dashboard reports, or if you link to a saved result set. You could run the search directly if that's the case.

I'd start with ensuring the regex being used is actually correct. Find the regex created in the props or transforms and add it to your search using the rex command.

[Drainy]

In 4.3 splunk will reload the search time field extractions when you run a search, I have been testing it a lot this week and it has worked 100% of the time! 🙂

[mikelanghorst]

you could use the command: | extract reload=T

But I've seen that will still take a bit before the new fields show up when I modify the config files directly.

[peasead]

The regex is right.

By "saved" I meant that the fields were cached (or something). It isn't actually a "saved search" in the traditional Splunk ideology.

Is there a way to refresh the extracted fields so that they look for new ones?