Does reason exist that btool does not return the inputs.conf stanza for the Splunk log folder (/opt/splunk/var/log/splunk)?

[jeffrey_berry]

Does a reason exist that btool does not return the inputs.conf stanza for the Splunk log folder (/opt/splunk/var/log/splunk)?

[splunk@localhost bin]$ ./splunk btool inputs list monitor:///opt/splunk/var/log/splunk
[splunk@localhost bin]$

It looks like a bug to me since the using the keyword "monitor" returns the stanza for it.

[splunk@localhost bin]$ ./splunk btool inputs list monitor | grep "\["
[monitor:///Library/Logs]
[monitor:///etc]
[monitor:///home/*/.bash_history]
[monitor:///opt/splunk/etc/splunk.version]
[monitor:///opt/splunk/var/log/introspection]
[monitor:///opt/splunk/var/log/splunk]
[monitor:///opt/splunk/var/log/splunk/license_usage_summary.log]
[monitor:///root/.bash_history]
[monitor:///var/adm]
[monitor:///var/log]
[splunk@localhost bin]$

[darrenk_splunk · Accepted solution]

It's not a bug but is easy to trip up on. The underlying stanza you're trying to find actually uses $SPLUNK_HOME as part of its path. When you specify the stanza name using btool you'll need to write it like below, i.e as the setting appears in-file, not the expanded version:

./splunk btool inputs list 'monitor://$SPLUNK_HOME/var/log/splunk'

This can be a bit confusing since if you specify simply ./splunk btool inputs list Splunk/btool will automatically expand $SPLUNK_HOME to the install dir for output, to help you understand the absolute path.

[jeffrey_berry]

Thanks @darrenk_splunk . Based on the info that you provided, I agree...it is not a bug. The Linux "cat" and grep commands help explain the unexpected output also.

[root@localhost bin]$ cat /opt/splunk/etc/system/default/inputs.conf | grep "monitor://"
[monitor://$SPLUNK_HOME/var/log/splunk]
[monitor://$SPLUNK_HOME/var/log/watchdog/watchdog.log*]
[monitor://$SPLUNK_HOME/var/log/splunk/license_usage_summary.log]
[monitor://$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*]
[monitor://$SPLUNK_HOME/etc/splunk.version]
[root@localhost bin]$ 

[bandit]

I'm not able to recreate your issue. Splunk is monitoring $SPLUNK_HOME/var/log/splunk out of the box. You can also use the the --debug switch to show the full path to the conf file. i.e. ./splunk btool --debug inputs list

/opt/spl/splunk/bin/splunk btool --debug inputs list | grep "var/log"
/opt/spl/splunk/etc/apps/introspection_generator_addon/default/inputs.conf [monitor:///opt/spl/splunk/var/log/introspection]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk/license_usage_summary.log]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk/splunk_instrumentation_cloud.log*]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/watchdog/watchdog.log*]

_

/opt/spl/splunk/bin/splunk btool --debug inputs list | grep "\["
/opt/spl/splunk/etc/system/default/inputs.conf                             [SSL]
/opt/spl/splunk/etc/system/default/inputs.conf                             [batch:///opt/spl/splunk/var/run/splunk/search_telemetry/*search_telemetry.json]
/opt/spl/splunk/etc/system/default/inputs.conf                             [batch:///opt/spl/splunk/var/spool/splunk]
/opt/spl/splunk/etc/system/default/inputs.conf                             [batch:///opt/spl/splunk/var/spool/splunk/...stash_new]
/opt/spl/splunk/etc/system/default/inputs.conf                             [blacklist:/opt/spl/splunk/etc/auth]
/opt/spl/splunk/etc/system/default/inputs.conf                             [blacklist:/opt/spl/splunk/etc/passwd]
/opt/spl/splunk/etc/system/default/inputs.conf                             [fschange:/opt/spl/splunk/etc]
/opt/spl/splunk/etc/apps/splunk_httpinput/default/inputs.conf              [http]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/etc/splunk.version]
/opt/spl/splunk/etc/apps/introspection_generator_addon/default/inputs.conf [monitor:///opt/spl/splunk/var/log/introspection]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk/license_usage_summary.log]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/splunk/splunk_instrumentation_cloud.log*]
/opt/spl/splunk/etc/system/default/inputs.conf                             [monitor:///opt/spl/splunk/var/log/watchdog/watchdog.log*]
/opt/spl/splunk/etc/system/default/inputs.conf                             [script]
/opt/spl/splunk/etc/apps/introspection_generator_addon/default/inputs.conf [script:///opt/spl/splunk/etc/apps/introspection_generator_addon/bin/collector.path]
/opt/spl/splunk/etc/apps/splunk_instrumentation/default/inputs.conf        [script:///opt/spl/splunk/etc/apps/splunk_instrumentation/bin/instrumentation.py]
/opt/spl/splunk/etc/apps/splunk_instrumentation/default/inputs.conf        [script:///opt/spl/splunk/etc/apps/splunk_instrumentation/bin/on_splunk_start.py]
/opt/spl/splunk/etc/apps/splunk_instrumentation/default/inputs.conf        [script:///opt/spl/splunk/etc/apps/splunk_instrumentation/bin/schedule_delete.py]
/opt/spl/splunk/etc/apps/splunk_monitoring_console/default/inputs.conf     [script:///opt/spl/splunk/etc/apps/splunk_monitoring_console/bin/dmc_config.py]
/opt/spl/splunk/etc/system/default/inputs.conf                             [splunktcp]
/opt/spl/splunk/etc/system/default/inputs.conf                             [tcp]
/opt/spl/splunk/etc/system/default/inputs.conf                             [udp]

[jeffrey_berry]

@rob_jordan To re-create the issue, you have to include the name of the stanza with the btool command.

[splunk@localhost bin]$ ./splunk btool inputs list monitor:///opt/splunk/var/log/splunk
[splunk@localhost bin]$

Based on the output in your post, the stanza name would be monitor:///opt/spl/splunk/var/log/splunk for your Splunk instance.

[bandit]

Ok @jeffrey_berry I see you are using the the optional stanzaPrefix which I had not known existed. For me it works up to monitor:// then returns nothing if I add monitor:///

I don't see any good examples in the docs so not sure if it's designed to match the entire stanza. I've always used grep to filter my results. Are you trying to return something specific from the config or just reporting a bug or possible enhancement?

./splunk btool
Usage:
        btool [options] CONF_FILE {list|layer|add|delete} [stanzaPrefix]
Usage:
        btool [options] CONF_FILE {list|layer|add|delete} [stanzaPrefix]
        btool [options] {check|validate-strptime|validate-regex}
        btool [options]Options:
        --debug
        --debug-logfile=FILENAME
        --debug-print=[user|app|stanza|sourcefile]
        --user=SPLUNK_USERNAME
        --app=SPLUNK_APP
        --dir=ETC_DIR
        --searchpool=SEARCHPOOL_DIR
        --slave-apps=SLAVE_APPS
        --peername=SEARCH_PEER_NAME
        --expand-stanzas=[true|false]

[jeffrey_berry]

@rob_jordan With your confirmation and confirmation from other users, I am just reporting a possible bug (update 1/21/2020: not a bug...see answer below), and spreading awareness that the btool may not return the expected output for certain input. My question here was worded in the off-chance that it is not a bug, and the output could be explained. Per a recent Data Admin training class, the entire stanza name can be included in the btool command. For another example, the "monitor:///var/log" stanza (i.e. entire stanza name) returns the expected output (see below). However, certain stanzas in the default inputs.conf file do not return the expected output.

I am aware that it appears that using grep is a work around. However, I would think that you would agree that It is inconsistent behavior of the btool command, and Splunk users should be aware of it.

[root@localhost bin]$ ./splunk btool inputs list monitor:///var/log
[monitor:///var/log]
_rcvbuf = 1572864
blacklist = (lastlog|anaconda.syslog)
disabled = false
host = localhost.localdomain
index = default
whitelist = (.log|log$|messages|secure|auth|mesg$|cron$|acpid$|.out)
[root@localhost bin]$

[BainM]

Hey Jeffrey-
That's an odd return response. What type of box are you running that on? I just tried that on a deployment server/SH and did not get the /var/log path, but only the following:
[monitor:///opt/splunk/etc/splunk.version]
[monitor:///opt/splunk/var/log/introspection]
[monitor:///opt/splunk/var/log/splunk]
[monitor:///opt/splunk/var/log/splunk/license_usage_summary.log]
[monitor:///opt/splunk/var/log/watchdog/watchdog.log*]

BTW, your grep of the "[" did not work for me. I got a regex error.

[jeffrey_berry]

@BainM Splunk Answers website removed the backslash character in the second btool comand. I was able to edit my question, and add another backslash to get one backslash to appear. Adding the backslash should fix the regex error. I am using a CentOS VM.

I agree...it is an odd response from btool. Other stanzas with a /opt/splunk/var/log parent path are not returned by "splunk btool inputs list" command also using the entire stanza name.