topic Re: Can't See Newly Creating Fields in Splunk Search

Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 16:41:14 GMT

I just created a new search field name going through the following process;

1.  Run a simple search
2.  Select “Extract Fields”
3.  Edit the regex & run a “test” to verify that it works, save it and give it a name

Then I review the Manager>Fields>Field extractions web page searching on “App context” = Search (search) and “Owner” = Me, and there it is.

Name                              Type     Extraction/Transform                             Owner   App      Sharing              Status    Actions
crm_cid_log : EXTRACT-CPC_ACCTNO  Inline   (?i)<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)  myname  search  Global | Permissions   Enabled  Move | Delete

Under permissions I have “All apps” selected and under “Roles” I have Everyone Read & Write.

Now, when I go back and run the same search, on the left side on the Web page I do not see the field name. When I go into the “View all nn fields”, my new field is not there either.

Can anyone give me an idea of what’s going on?

~Ed

Re: Can't See Newly Creating Fields

aholzer — Mon, 13 Jan 2014 18:47:23 GMT

Try the following:

| extract reload=T

This should force Splunk to reload your field definitions and run them again.

I've noticed that sometimes it takes Splunk a while to recognize a new field definition.

Hope this helps

Re: Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 18:52:34 GMT

I tried the | extract reload=T and, unfortunately, no luck. Thanks anyway.
~Ed

Re: Can't See Newly Creating Fields

somesoni2 — Mon, 13 Jan 2014 19:59:42 GMT

This has helped me a lot of time with similar issue. Just restart your Splunk Instance and see if those fields are available. This is not a standard solution, but may work for you.

Re: Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 20:02:10 GMT

That was the first thing that I tried was the restart. Unfortunately it was no help this time.
~Ed

Re: Can't See Newly Creating Fields

lguinn2 — Mon, 13 Jan 2014 21:36:14 GMT

Did you follow these guidelines for field names? Field names should contain only letters, numbers and underscores. They must start with a letter. I know that you can use field names with spaces in them - but I have found that these guidelines work in all contexts and without quotation marks.
Do all of the events have this field? I assume that the answer to this is yes, because you ran the same simple search twice. But what happens if you search for CPC_ACCTNO=*
Remember that field names are case-sensitive
The fields sidebar (and even the "show all fields" popup window) have thresholds - a field must be present in a minimum % of events in order to appear in the list.

Also, when I look at this:

(?i)<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)

I see a possible problem with the regular expression. Edit the regular expression to match the following and it might help - if it does, there might be a bug in the field extractor:

(?i)\<BILLINGACCOUNTNUMBER>(?P<CPC_ACCTNO>[^<]+)

(see the backslash (\) that I put as the 5th character?)

Re: Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 21:36:27 GMT

OK, follow-up newbie question. I haven't seen any log files come in to the system since I created the new fields. Do I have to wait for something new to come in or should the fields be there once I create the new fields?

Re: Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 21:53:27 GMT

Yes
Most do.
Understood.
Understood () added with no change.

Now, for my newbie question from above. I haven't seen any events come in to the system since I created the new fields. Do I have to wait for something new to come in or should the fields be there from the old data once I create the new fields?

Re: Can't See Newly Creating Fields

OldManEd — Mon, 13 Jan 2014 21:54:21 GMT

Number 4 should read "Understood, backslash added with no change." Sorry.

Re: Can't See Newly Creating Fields

lguinn2 — Tue, 14 Jan 2014 16:19:10 GMT

One of the wonderful things about fields is that they are extracted at search time - so they apply to all data, old and new.

So yes, you should be seeing your fields.

I just feel that we are overlooking something obvious here. I wonder if we are looking at the wrong things - can we see

1 - a sample of the data

2 - the search that you ran

Re: Can't See Newly Creating Fields

OldManEd — Tue, 14 Jan 2014 17:04:37 GMT

OK, an update here. I tried to run the following query;

index= sourcetype="" | head 10000 | rex "(?i)(?P[^<]+)" | top 50 CPC_ACCTNO

And it worked like a champ.

I don't know what's going on here.

Re: Can't See Newly Creating Fields

OldManEd — Fri, 17 Jan 2014 22:08:22 GMT

Update: I just discovered that the index associated with this search is a "summary index". My question now is does this new information affect the process of creating fields in any way?