Splunk Search

Why is my newly created field extraction not showing up in the fields sidebar?

martywalser
Explorer

I've seen similar questions to mine asked, but none of the advice has solved my issue.

I created a new field extraction (which correctly pulled the data in the 10,000 event sampling) and it shows up in the field extraction list under "Settings":

API Server Logs - 03-12 : EXTRACT-MW_ErrorMessage   Inline  (Error Code=").+>(?P<MW_ErrorMessage>[^<]+) mwalser    search    Private | Permissions  Enabled

After creating the field extraction, I attempted to reload the search:

index="cisres_events" sourcetype="API Server Logs - 03-12" | extract reload=T

But the newly created extraction "MW_ErrorMessage" does not show up in the selectable list of "All Fields".

What else might be causing the field extraction to not show up in the list? I've attempted to rebuild this extraction several times with different naming conventions and even tried modifying the permissions to no avail. Any suggestions?

1 Solution

martywalser
Explorer

So, it was a small UI foible that was hiding my newly created fields. In the field list there is a "coverage" drop down. "Coverage 1% or more" was selected value, but the extracts I had created only showed up in 0.8% of the logs I was exploring. It appears 1% is the default since I hadn't changed this value prior to discovering my issue.

Select &#39;All fields&#39;

View solution in original post

martywalser
Explorer

So, it was a small UI foible that was hiding my newly created fields. In the field list there is a "coverage" drop down. "Coverage 1% or more" was selected value, but the extracts I had created only showed up in 0.8% of the logs I was exploring. It appears 1% is the default since I hadn't changed this value prior to discovering my issue.

Select &#39;All fields&#39;

woodcock
Esteemed Legend

Is your search mode set to Verbose (not Fast, not Smart )?
A field is only interesting if it occurs in 95ish% (I forget the exact number) of events that are returned in the search. If your field is very rare, it is not interesting and will not show up. You can however select it from the Field Picker and that makes it show up as Selected.
Also, have you hit the _bump endpoint to refresh your session?

janderson19
Path Finder

it's been a much longer day than I originally thought

0 Karma

martywalser
Explorer

It turns out the the Field Picker was showing "Coverage 1% or more" and not "All Fields". These extractions were only occurring in about 0.8% of records, so they were not showing up on the 1% list.

I have added this as the answer but it has not been moderated yet.

0 Karma

woodcock
Esteemed Legend

So I got you 90+% to the answer!

0 Karma

martywalser
Explorer

The breadcrumbs got me there eventually. 🙂 I had not made the connection between the "interesting" metric and the coverage in the drop down. Honestly, I had never really noticed that "coverage" values before because other extractions I had created showed up in the majority of records, so it was never an issue until I started hunting for rare instances.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Fields are relative to sourcetype, are you searching on the same sourcetype you used when creating the field?

Another thing to note, field extractions take a minute or two to appear after creating them

0 Karma

somesoni2
SplunkTrust
SplunkTrust

What is the sharing permission on the field extractions? Is it private, This app or Global? If it's private or This App, when you're searching, are you in the same app context?

0 Karma

martywalser
Explorer

It was previously "private" but I have modified it to "App (search)" and in both instances it has not shown up. How do I know if I am in the same "app context"?

When I experienced this before, I went directly from the field creation back to the Search by clicking on the link that reads "Explore using this field in search" (or words to that effect). I.E. - I had used the back link that takes one back to the search from which the new extraction is created, and it still did not show up.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

If you created the field extraction in search, then if it's private OR this App, then you should be searching from Search app only. Also, did you verify that the regex works fine? May be try to run like this to see if the regex is correct.

index="cisres_events" sourcetype="API Server Logs - 03-12" | rex "(Error Code=").+>(?P<MW_ErrorMessage>[^<]+)"

Not sure if the regex got changed when you posted here, but it seems off. Try this for regex.

(Error Code=\").+\>(?P<MW_ErrorMessage>[^\<]+)

For more accurate suggestions, share your sample event from where you're trying to extract the field.

0 Karma

martywalser
Explorer

Thanks for the regex heads up. Yeah, I did have an error from a bad edit, but that wasn't the issue. I posted the answer below,

Thanks again for your help!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...