Getting Data In

Why are the indexers trying to execute these command if they are defined as 'local = true'?

Contributor

We've had some custom commands defined on our indexers for years. Here is /opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf:

[netbotzreport]
filename = netbotzreport.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
# this should not be necessary
overrides_timeorder = true
required_fields=mib,oid,snmp_index,value

[netbotzextract]
filename = netbotzextract.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
# this should not be necessary
overrides_timeorder = true

[pipesniff]
filename = pipesniff.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
Sometime in the last month, searches using these commands have started failing with these messages from the indexers:

[awnulsplunkp1] Search Factory: Unknown search command 'netbotzextract'.

We did a 6.5 -> 7.0 last week, which I suspect is what changed.

Why are the indexers trying to execute these command if they are defined as 'local = true'?

0 Karma

Motivator

Hey wegscd,

Any customization that is done has to be done in /opt/splunk/etc/apps/whirlpool_netbotz/local/commands.conf that is local and not in default directory.
The changes that were done in default directory got overwritten after the upgrade.

Create a commands.conf file in local directory in your app and add the changes there.
And you can cross check what configs are used by indexer by running following command on indexer

/$SPLUNK_HOME$/bin/splunk cmd btool commands list --debug

0 Karma

Contributor

there is nothing in local/ to override default/commands,conf, and nothing there got overwritten in the upgrade. The btool says that the local = true in default is being used.

/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         [netbotzextract]
/opt/splunk/etc/system/default/commands.conf                         changes_colorder = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         enableheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         filename = netbotzextract.py
/opt/splunk/etc/system/default/commands.conf                         generates_timeorder = false
/opt/splunk/etc/system/default/commands.conf                         generating = false
/opt/splunk/etc/system/default/commands.conf                         is_risky = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         local = true
/opt/splunk/etc/system/default/commands.conf                         maxinputs = 50000
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         outputheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         overrides_timeorder = true
/opt/splunk/etc/system/default/commands.conf                         passauth = false
/opt/splunk/etc/system/default/commands.conf                         perf_warn_limit = 0
/opt/splunk/etc/system/default/commands.conf                         required_fields = *
/opt/splunk/etc/system/default/commands.conf                         requires_preop = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         requires_srinfo = true
/opt/splunk/etc/system/default/commands.conf                         retainsevents = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         stderr_dest = message
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         streaming = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_getinfo = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_multivalues = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_rawargs = true
/opt/splunk/etc/system/default/commands.conf                         type = python
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         [netbotzreport]
/opt/splunk/etc/system/default/commands.conf                         changes_colorder = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         enableheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         filename = netbotzreport.py
/opt/splunk/etc/system/default/commands.conf                         generates_timeorder = false
/opt/splunk/etc/system/default/commands.conf                         generating = false
/opt/splunk/etc/system/default/commands.conf                         is_risky = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         local = true
/opt/splunk/etc/system/default/commands.conf                         maxinputs = 50000
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         outputheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         overrides_timeorder = true
/opt/splunk/etc/system/default/commands.conf                         passauth = false
/opt/splunk/etc/system/default/commands.conf                         perf_warn_limit = 0
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         required_fields = mib,oid,snmp_index,value
/opt/splunk/etc/system/default/commands.conf                         requires_preop = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         requires_srinfo = true
/opt/splunk/etc/system/default/commands.conf                         retainsevents = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         stderr_dest = message
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         streaming = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_getinfo = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_multivalues = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_rawargs = true
/opt/splunk/etc/system/default/commands.conf                         type = python
0 Karma

Path Finder

I am having the same problem.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!