Getting Data In

Why are the indexers trying to execute these command if they are defined as 'local = true'?

wegscd
Contributor

We've had some custom commands defined on our indexers for years. Here is /opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf:

[netbotzreport]
filename = netbotzreport.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
# this should not be necessary
overrides_timeorder = true
required_fields=mib,oid,snmp_index,value

[netbotzextract]
filename = netbotzextract.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
# this should not be necessary
overrides_timeorder = true

[pipesniff]
filename = pipesniff.py
enableheader = true
outputheader = true
requires_srinfo = true
stderr_dest = message
supports_getinfo = true
supports_rawargs = true
supports_multivalues = true
local = true
streaming = true
Sometime in the last month, searches using these commands have started failing with these messages from the indexers:

[awnulsplunkp1] Search Factory: Unknown search command 'netbotzextract'.

We did a 6.5 -> 7.0 last week, which I suspect is what changed.

Why are the indexers trying to execute these command if they are defined as 'local = true'?

0 Karma

deepashri_123
Motivator

Hey wegscd,

Any customization that is done has to be done in /opt/splunk/etc/apps/whirlpool_netbotz/local/commands.conf that is local and not in default directory.
The changes that were done in default directory got overwritten after the upgrade.

Create a commands.conf file in local directory in your app and add the changes there.
And you can cross check what configs are used by indexer by running following command on indexer

/$SPLUNK_HOME$/bin/splunk cmd btool commands list --debug

0 Karma

wegscd
Contributor

there is nothing in local/ to override default/commands,conf, and nothing there got overwritten in the upgrade. The btool says that the local = true in default is being used.

/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         [netbotzextract]
/opt/splunk/etc/system/default/commands.conf                         changes_colorder = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         enableheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         filename = netbotzextract.py
/opt/splunk/etc/system/default/commands.conf                         generates_timeorder = false
/opt/splunk/etc/system/default/commands.conf                         generating = false
/opt/splunk/etc/system/default/commands.conf                         is_risky = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         local = true
/opt/splunk/etc/system/default/commands.conf                         maxinputs = 50000
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         outputheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         overrides_timeorder = true
/opt/splunk/etc/system/default/commands.conf                         passauth = false
/opt/splunk/etc/system/default/commands.conf                         perf_warn_limit = 0
/opt/splunk/etc/system/default/commands.conf                         required_fields = *
/opt/splunk/etc/system/default/commands.conf                         requires_preop = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         requires_srinfo = true
/opt/splunk/etc/system/default/commands.conf                         retainsevents = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         stderr_dest = message
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         streaming = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_getinfo = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_multivalues = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_rawargs = true
/opt/splunk/etc/system/default/commands.conf                         type = python
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         [netbotzreport]
/opt/splunk/etc/system/default/commands.conf                         changes_colorder = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         enableheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         filename = netbotzreport.py
/opt/splunk/etc/system/default/commands.conf                         generates_timeorder = false
/opt/splunk/etc/system/default/commands.conf                         generating = false
/opt/splunk/etc/system/default/commands.conf                         is_risky = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         local = true
/opt/splunk/etc/system/default/commands.conf                         maxinputs = 50000
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         outputheader = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         overrides_timeorder = true
/opt/splunk/etc/system/default/commands.conf                         passauth = false
/opt/splunk/etc/system/default/commands.conf                         perf_warn_limit = 0
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         required_fields = mib,oid,snmp_index,value
/opt/splunk/etc/system/default/commands.conf                         requires_preop = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         requires_srinfo = true
/opt/splunk/etc/system/default/commands.conf                         retainsevents = false
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         stderr_dest = message
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         streaming = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_getinfo = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_multivalues = true
/opt/splunk/etc/apps/whirlpool_netbotz/default/commands.conf         supports_rawargs = true
/opt/splunk/etc/system/default/commands.conf                         type = python
0 Karma

kiril123
Path Finder

I am having the same problem.

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...