Getting Data In

Is there a config file checker?

lguinn2
Legend

If there isn't, how do I find the errors in my configuration files? What index & search string should I use to find config file problems?

Tags (1)
1 Solution

ewoo
Splunk Employee
Splunk Employee

As of 4.2, there is a conf checker that runs on "splunk start". You can manually invoke it with the following command:

./splunk btool check

Any typos will be printed to stdout:

$ cat $SPLUNK_HOME/etc/system/local/indexes.conf
[hello]
# This should be capitalized "homePath".
homepath = $SPLUNK_DB/hello/db

$ splunk btool check
Possible typo in stanza [hello] in /opt/splunk/etc/system/local/indexes.conf, line 3: homepath  =  $SPLUNK_DB/hello/db

View solution in original post

lguinn2
Legend

Two new things:

1 - Splunk now explicitly checks your configuration files (using btool) as part of the standard startup. So always check the Splunk messages at startup.

2 - The SOS app (Splunk on Splunk) is very helpful for finding your configuration errors. Download it free from Splunkbase here.

2017 update (because answers never die!)
btool is still a wonderful thing. But instead of the SOS app, use the built-in Splunk Monitoring Console. (It is newer and it is the offspring of the SOS app.)

And, even in older versions of Splunk, remember that Splunk has its own internal logs, which it indexes into _internal.
So you can search

index=_internal error OR warn*

And see all the error messages that Splunk has logged. But if the problem means that Splunk was unable to index, you may need to examine the internal log files directly. splunkd.log is your friend. Use grep or findstr or some tool to find the errors and warnings.

0 Karma

ewoo
Splunk Employee
Splunk Employee

As of 4.2, there is a conf checker that runs on "splunk start". You can manually invoke it with the following command:

./splunk btool check

Any typos will be printed to stdout:

$ cat $SPLUNK_HOME/etc/system/local/indexes.conf
[hello]
# This should be capitalized "homePath".
homepath = $SPLUNK_DB/hello/db

$ splunk btool check
Possible typo in stanza [hello] in /opt/splunk/etc/system/local/indexes.conf, line 3: homepath  =  $SPLUNK_DB/hello/db

BobM
Builder

I've voted this answer up but maybe Dan or someone with edit priv can combine the two as they are both using btool.

0 Karma

Dan
Splunk Employee
Splunk Employee

I'm not aware of any config validation tool (for typos and such). The closest thing is the CLI btool command, which is documented here: http://docs.splunk.com/Documentation/Splunk/5.0/Troubleshooting/Usebtooltotroubleshootconfigurations

Troubleshooting configurations

Splunk's configuration file system supports many overlapping configuration files in many different locations. How these configuration files interact with and take precedence over one another is described in "Configuration file precedence" in this manual. Sometimes the price of this level of flexibility is that figuring out which value for which configuration option is being used in your Splunk installation.

Splunk provides a command line tool you can use to help troubleshoot issues with your configuration files, or just see what values are being used by your Splunk installation.

The command line tool is called btool. You can run btool for a given App in your Splunk installation, and it will list out all the configuration options that are currently in use by that App for a given configuration file.

To run btool, go to $SPLUNK_HOME/bin and type:

./splunk cmd btool App_name list prefix

where App_name is the name of the App you want to see the configurations for and prefix is the name of the config file you're interested in without the .conf extension. list indicates that you want to list the options. btool supports changing configuration files as well, but Splunk does not recommend you use btool this way without discussing it with the Splunk Support team.

So for example, if you want to know what configuration options are being used in props.conf by the Search App, you'd specify the Search App and props.conf in your btool string like so:

./splunk cmd btool Search list [props]

and would see a list of the props.conf settings currently being used for the Search App.

You can then save this list to a file and examine it.

Note: btool is not tested by Splunk and is not officially supported or guaranteed. That said, this is what our Support team uses when trying to troubleshoot your issues.

lguinn2
Legend

What if you want to examine the configurations specified in the $SPLUNK_HOME/etc/system/local directory?

(and Thanks Dan!)

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...