I'm not wanting to monitor these events and yet the scripts still run every minute (by looking at the task manager). I have tried adding stanzas to disable them in etc/system/local/inputs.conf but this doesn't seem to work, the still run.
[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path] disabled = 1 [script://$SPLUNK_HOME\bin\scripts\splunk-netmon.path] disabled = 1 [script://$SPLUNK_HOME\bin\scripts\splunk-winprintmon.path] disabled = 1 [script://$SPLUNK_HOME\bin\scripts\splunk-powershell.path] disabled = 1 [script://$SPLUNK_HOME\bin\scripts\splunk-powershell2.path] disabled = 1 [WinRegMon] disabled=1 [WinNetMon] disabled=1 [WinPrintMon] disabled=1 [perfmon] disabled=1 [admon] disabled=1 [powershell] disabled=1 [powershell2] disabled=1
I can see in the splunk.log file the following
INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-MonitorNoHandle.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-netmon.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-perfmon.exe" INFO ExecProcessor - interval: run once INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe" --ps2 INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-regmon.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe" INFO ExecProcessor - interval: 60000 ms INFO ExecProcessor - New scheduled exec process: "C:\Program Files\SplunkUniversalForwarder\bin\splunk-winprintmon.exe" INFO ExecProcessor - interval: 60000 ms
Here's my btool inputs list --debug output for lsof:
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf disabled = 1
/opt/splunkforwarder/etc/system/local/inputs.conf host = c20sbap01l01
/opt/splunkforwarder/etc/apps/DS2-ns2-Splunk_TA_nix-cre/local/inputs.conf index = cre_linux
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf interval = 600
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf source = lsof
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf sourcetype = lsof
Here's my splund.log output:
10-10-2019 16:07:12.898 +0000 INFO ExecProcessor - New scheduled exec process: /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/lsof.sh
10-10-2019 16:07:12.898 +0000 INFO ExecProcessor - interval: 60000 ms
I've tried restarting splunk to no effect...
Notice that the interval is set to 600 (600 seconds) in the btool output, but 60000 (60 seconds) in the splunkd.log output.
I'll try interval = -1 next.
Btool is your friend:
./splunk cmd btool YOURFILENAME list --debug
Simply try to debug one of those stanzas and see where the parameters are coming from.
If you can't find out what's going on, post the relevant bit of your btool output here and we'll try to help.
Here is one of the stanzas
C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf [WinNetMon] C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf _rcvbuf = 1572864 C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf baseline = 0 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf dedicatedIoThreads = 2 C:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf disabled = 1 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf enableSSL = 1 C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dc_name = C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_dns_name = C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf evt_resolve_ad_obj = 0 host = ODIN index = default C:\Program Files\SplunkUniversalForwarder\etc\system\default\inputs.conf interval = 60 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf maxSockets = 0 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf maxThreads = 0 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf port = 8088 C:\Program Files\SplunkUniversalForwarder\etc\apps\splunk_httpinput\default\inputs.conf useDeploymentServer = 0
Are you not supposed to specify your network monitoring input after the WinNetMon stanza?
Take a look at the syntax from the inputs.conf spec file:
[WinNetMon://<name>] * This section explains possible attribute/value pairs for configuring Splunk's Network Monitor. * Each WinNetMon:// stanza represents an individually configured network monitoring input. The value of "$NAME" will match what was specified in Splunk Web. Splunk recommends that you use the Manager interface to configure Network Monitor inputs because it is easy to mistype the values for Network Monitor monitor objects, counters and instances.
Based on that logic, can you double check with btool there's no other WinNetMon stanza?
Same goes for perfmon, etc. See below:
# Perfmon: Windows performance monitoring examples # Important: You must specify the names of objects, counters and instances # exactly as they are shown in the Performance Monitor application. Splunk Web # is the recommended interface to use to configure performance monitor inputs. # Important: These stanzas gather performance data from the local system only. # Use wmi.conf for performance monitor metrics on remote systems. # Query the PhysicalDisk performance object and gather disk access data for # all physical drives installed in the system. Store this data in the # "perfmon" index. # Note: If the interval attribute is set to 0, Splunk will reset the interval # to 1. [perfmon://LocalPhysicalDisk] interval = 0 object = PhysicalDisk counters = Disk Bytes/sec; % Disk Read Time; % Disk Write Time; % Disk Time instances = * disabled = 0 index = PerfMon # Gather common memory statistics using the Memory performance object, every # 5 seconds. Store the data in the "main" index. Since none of the counters # specified have applicable instances, the instances attribute is not required. [perfmon://LocalMainMemory] interval = 5 object = Memory counters = Committed Bytes; Available Bytes; % Committed Bytes In Use disabled = 0 index = main # Gather data on USB activity levels every 10 seconds. Store this data in the default index. [perfmon://USBChanges] interval = 10 object = USB counters = Usb Control Data Bytes/Sec instances = * disabled = 0
I guess if you want to disable this stanza you have to do it individually by using the whole stanza name and not just perfmon.
Let me know if that helps.
Run the following in order to capture everything in a file:
./splunk cmd btool inputs list --debug > /tmp/btool_inputs.txt
Then inspect the file and try to identify every instance of those stanzas. If you want to disable any of these you need to match exactly the stanza name in your local inputs.conf and specify the disabled = 1 below.
I'm running out of ideas in that case. I'm assuming you already tried to restart the UF after all those changes and that didn't help so I would suggest you open a support call with Splunk and let them investigate.