Splunk Search

Can't See Newly Creating Fields

OldManEd
Builder

I just created a new search field name going through the following process;

1.  Run a simple search
2.  Select “Extract Fields”
3.  Edit the regex & run a “test” to verify that it works, save it and give it a name

Then I review the Manager>Fields>Field extractions web page searching on “App context” = Search (search) and “Owner” = Me, and there it is.

Name                              Type     Extraction/Transform                             Owner   App      Sharing              Status    Actions
crm_cid_log : EXTRACT-CPC_ACCTNO  Inline   (?i)<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)  myname  search  Global | Permissions   Enabled  Move | Delete

Under permissions I have “All apps” selected and under “Roles” I have Everyone Read & Write.

Now, when I go back and run the same search, on the left side on the Web page I do not see the field name. When I go into the “View all nn fields”, my new field is not there either.

Can anyone give me an idea of what’s going on?

~Ed

Tags (2)
0 Karma

OldManEd
Builder

OK, follow-up newbie question. I haven't seen any log files come in to the system since I created the new fields. Do I have to wait for something new to come in or should the fields be there once I create the new fields?

0 Karma

lguinn2
Legend
  1. Did you follow these guidelines for field names? Field names should contain only letters, numbers and underscores. They must start with a letter. I know that you can use field names with spaces in them - but I have found that these guidelines work in all contexts and without quotation marks.

  2. Do all of the events have this field? I assume that the answer to this is yes, because you ran the same simple search twice. But what happens if you search for CPC_ACCTNO=*

  3. Remember that field names are case-sensitive

  4. The fields sidebar (and even the "show all fields" popup window) have thresholds - a field must be present in a minimum % of events in order to appear in the list.

Also, when I look at this:

(?i)<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)

I see a possible problem with the regular expression. Edit the regular expression to match the following and it might help - if it does, there might be a bug in the field extractor:

(?i)\<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)

(see the backslash (\) that I put as the 5th character?)

0 Karma

OldManEd
Builder

Update: I just discovered that the index associated with this search is a "summary index". My question now is does this new information affect the process of creating fields in any way?

0 Karma

OldManEd
Builder

OK, an update here. I tried to run the following query;

index= sourcetype="" | head 10000 | rex "(?i)(?P[^<]+)" | top 50 CPC_ACCTNO

And it worked like a champ.

I don't know what's going on here.

0 Karma

lguinn2
Legend

One of the wonderful things about fields is that they are extracted at search time - so they apply to all data, old and new.

So yes, you should be seeing your fields.

I just feel that we are overlooking something obvious here. I wonder if we are looking at the wrong things - can we see

1 - a sample of the data

2 - the search that you ran

0 Karma

OldManEd
Builder

Number 4 should read "Understood, backslash added with no change." Sorry.

0 Karma

OldManEd
Builder
  1. Yes
  2. Most do.
  3. Understood.
  4. Understood () added with no change.

Now, for my newbie question from above. I haven't seen any events come in to the system since I created the new fields. Do I have to wait for something new to come in or should the fields be there from the old data once I create the new fields?

0 Karma

OldManEd
Builder

That was the first thing that I tried was the restart. Unfortunately it was no help this time.
~Ed

0 Karma

somesoni2
SplunkTrust
SplunkTrust

This has helped me a lot of time with similar issue. Just restart your Splunk Instance and see if those fields are available. This is not a standard solution, but may work for you.

0 Karma

OldManEd
Builder

I tried the | extract reload=T and, unfortunately, no luck. Thanks anyway.
~Ed

0 Karma

aholzer
Motivator

Try the following:

| extract reload=T

This should force Splunk to reload your field definitions and run them again.

I've noticed that sometimes it takes Splunk a while to recognize a new field definition.

Hope this helps

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...