I can't figure out why lsof.sh is running every minute.
Here's the "btool inputs list --debug" output for lsof:
/opt/splunkforwarder/etc/apps/DS2-ns2-Splunk_TA_nix-cre/local/inputs.conf [script:///opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/lsof.sh] /opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864 /opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf disabled = 1 /opt/splunkforwarder/etc/system/local/inputs.conf host = c20sbap01l01 /opt/splunkforwarder/etc/apps/DS2-ns2-Splunk_TA_nix-cre/local/inputs.conf index = cre_linux /opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf interval = 600 /opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf source = lsof /opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf sourcetype = lsof
Here's my splund.log output:
10-10-2019 16:07:12.898 +0000 INFO ExecProcessor - New scheduled exec process: /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/lsof.sh 10-10-2019 16:07:12.898 +0000 INFO ExecProcessor - interval: 60000 ms
I've tried restarting splunk to no effect...
Notice that the interval is set to 600 (600 seconds) in the btool output, but 60000 (60 seconds) in the splunkd.log output.
I'll try interval = -1 next, and then a single app after that.
Do you want to stop lsof.sh from running at all or just make it run every 10 minutes?
Yes, I did. I have since stopped Splunk, removed deploymentclient.conf, and the DS2-SplunkTAnix-cre directory, and set the indexes for all the SplunkTAnix inputs in SplunkTAnix/local/inputs.conf, and restarted splunk. I'm still getting a steady stream of lsof events every minute.
Ok, I stopped Splunk, removed the SplunkTAnix app, started splunk put the app back and started splunk, and I'm finally no longer getting lsof events. However, I now need to do the same on all my deployment clients... Good thing I was planning on working late.
btool is your friend here. You could have an
inputs.conf in any app that is causing things so try this:
$SPLUNK_HOME/bin/splunk btool list inputs --debug | grep lsof