I wish to start using TLS and mutual authentication for my forwarders. I'm running Splunk Enterprise  8.2.4 on Windows 2019 with a mix of windows and Linux forwarders all on 8.2.5 I have my indexers using our PKI signed certificates and the forwarders are currently using the default UF certificate which is obviously not good. As such, I can get certificates issues from our PKI for the UFs but reading this guide it does not indicate what the key usage or enhanced key usage for the cert should be? I would have assumed client authentication ? Looking at the default certificates these attributes are not specified.  ... View more

I'm using Splunk Enterprise 8.2.5 on Windows and using deployment server to push apps. There is currently no indexer configured in /etc/system/local/outputs.conf as we do all this in the app.  Our security team want our forwarders to start shipping up the events over TLS 1.2. As a quick test I have the following with non-deployment server app: Indexer listening for TLS 1.2 using my PKI signed cert on TCP port 9998 Forwarder using default server.pem cert and verifying the indexer cert This works fine but now I have the issue of how to deploy apps from deployment server. If I use the default, a self-signed or PKI signed client cert at the forwarder I must secure the private key with a password and specify that password in the outputs.conf. Therefore if specifying the indexer for a given app (not all apps have the same indexer!) I need to specify the password for that app in the <app>/outputs.conf file at the deployment server. I tested this but one issue is the password does not get encrypted on the deployment server or the target UF. I'd realistically needs the same client certificate on all forwarders to make this manageable. Should I be defining all indexers outside of deployment server apps in /etc/system/local/outputs.conf and then routing to them in the <app>/outputs.conf instead? Note: Although it states here that if the password is specified in a conf file outside /etc/system/local/ it will not be encrypted I have tested and it is! This whole area of config is very confusing IMO 😞   ... View more

Using Splunk Enterprise 8.2.4 on Windows and trying to configure my forwarders to use SSL to forward events to my indexers. Client certificate verification is not enabled on the indexer. Reading the guide here it states some config aspects that don't seem to be true from testing: sslPassword = <password> * The password associated with the Certificate Authority certificate (CAcert). Is this  not the password for the client cert (server.pem by default) rather than CACert? Also, is states: useSSL = <true|false|legacy> * Whether or not the forwarder uses SSL to connect to the receiver, or relies on the 'clientCert' setting to be active for SSL connections. * You do not need to set 'clientCert' if 'requireClientCert' is set to "false" on the receiver. This appears to indicate that you can use SSL without a client certificate (which would be great!). However, if I simply add  useSSL=true to my forwarder outputs.conf the SSL connection does not come up and the following is showing splunkd.log indicating it is looking for a client certificate file: 03-30-2022 23:39:47.933 +0100 ERROR SSLCommon [31888 parsing] - Can't read certificate file errno=33558528 error:02001000:system library:fopen:system library My outputs.conf is as follows: [tcpout:test-ssl-1] disabled = 0 server = indexer1.mydomain.com:9998 useSSL = true useClientSSLCompression = true sslVerifyServerCert = false This outputs.conf does work: [tcpout:test-ssl-1] disabled = 0 server = indexer1.mydomain.com:9998 useSSL = true useClientSSLCompression = true sslVerifyServerCert = false sslCommonNameToCheck = indexer1 sslAltNameToCheck = indexer1.mydomain.com sslRootCAPath = C:\Program Files\SplunkUniversalForwarder\etc\auth\MyPKIChain.pem  I have no idea why the 2nd config works! sslVerifyServerCert  is false.    ... View more

I'm using Splunk Enterprise 8.2.4 and I would like to start getting my Windows Forwarder Estate (8.2.4) to send it's perform. Initially I thought this would be easy but I was wrong. I though that out of the box that Splunk would allow me collect Windows perfmon data straight to a metrics index.  I think from reading the guide here that the pattern is as follows: Configure the forwarder inputs stanza as normal i.e. as you would to collect say the CPU metrics to an events index Point it at a metrics index tagged with a custom sourcetype Transform/parse the event to metrics format at the indexer when received based on sourcetype Is this understanding correct and of so does anyone have a bundle of Transforms ready to go (perhaps a TA or app that does this like Splunk Add-on for Microsoft Windows | Splunkbase )? ... View more

I'm using Splunk Enterprise 8.2.4 and trying to get my forwarders to forward perfmon counters (CPU, Disk Space etc.) into a metrics index at my indexer cluster. It seems to me from reading here: This requires the Splunk App for Infrastructure (which is EOL I think?) or the Splunk AddOn for windows? This might affect existing collection of metrics that are being indexed as events This is a configuration done at the indexer and not the forwarder In short, I'm confused on how to achieve this! Any assistance would be much appreciated! ... View more

Is it possible to prevent a system admin adding inputs at a forwarder? I only want sanctioned inputs to be used i.e. I want our Splunk admins to approve all forwarders and inputs. I thought deployment server might solve this but it does not cover all local inputs per say.  ... View more

I'm using Splunk Enterprise 8.2.4 with deployment server. I wat to push out all config/apps to my forwarders to prevent server admins adding config/apps locally. To date system admins have been creating their own inputs and dumping data into main, flooding the license usages etc. and I need to stop this happening. I only want approved configs/inputs etc. to be pushed to the forwarders. As such, I have onboarded all my forwarders to deployment server. My first question is: Q1: How to prevent a user at the system creating an input and pushing data to the indexers? Is their a config item to only accept deployment server deployed inputs? On a test system I pushed an application I created that disabled the collection of the [WinEventLog://Security]. I found though that that system had received the app but was still pushing those events. Running btool at the forwarder shows: C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local\inputs.conf [WinEventLog://Security] C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local\inputs.conf disabled = 0 So this seems to be the config from when the forwarder was installed ad the windows inputs were selected in the forwarder MSI installation UI. Q2: How to override this with deployment server i.e. a locally configured input not necessarily in the apps folder?   ... View more