topic Re: Custom fields not always showing up in Splunk Search

Custom fields not always showing up

klumpba — Fri, 18 Jun 2010 02:45:02 GMT

I have a Splunk app that parses some Snort files and assigns some fields to the content. The app works fine from the Splunk web page, but when I use the REST API (via Splunk for Java) I do not get my custom fields back unless they are in the search criteria. For instance, if I search for "src_ip_addr=*" then I will get the fields back in the results, otherwise I don't.

I used to get them back regardless in Splunk 4.0. What do I need to change to get them to always come back for my sourcetype?

Thanks for any help...

-Brian

Re: Custom fields not always showing up

Lowell — Fri, 18 Jun 2010 02:55:42 GMT

Certain field extractions are automatic disabled in 4.1 when it appears that the field is not needed. (An interactive search from splunk's web interface all fields are enabled.)

I think all you should have to do is add a fields command to your search. Something like:

your search | fields src_ip_addr

You may also get the same effect by explicitly passing in the required_field_list to your dispatch request. (Was this just called fields in a previous version of the API? Maybe I'm remembering this incorrectly.) It seems like this should do the trick too, but I'm not 100% sure.

Re: Custom fields not always showing up

klumpba — Fri, 18 Jun 2010 22:57:50 GMT

Ah, thanks! Tossing "fields" in there did the trick. Thanks!

Re: Custom fields not always showing up

kbains — Thu, 30 Sep 2010 02:18:56 GMT

The other option is to use the required_field_list and status_bucket (anything over 0) during the POST:

status_buckets = integer

* defaults to 0.
* The most status buckets to generate. (0 = do not generate timeline information)

required_field_list = (comma separated list)

* default to empty
* This is the list of required fields that, even if not referenced or used directly by the search, will still be included by the events and summary endpoints.