Splunk 7.0.0 – GUI changes are not reflected in .conf files, deprecated paremters are returning

[dfollis]

Wondering if anyone else has seen issues where you modify the changes in the GUI and the applicable .conf files are not updated or modified incorrectly? This is Splunk 7 Enterprise running on Windows server 2016.

First example, spent far too much time trying to get LDAP authentication to work and gave up because we don't have a CA and Splunk didn't like the DC self-signed certs we are using, despite us importing them as trusted to the Windows Server and Splunk Trusted Root stores. Decided to use SAML via Okta instead, but while trying to figure that out, we noticed these errors in the log:

11-22-2017 18:09:35.434 -0800 WARN SSLOptions - authentication.conf/[saml]/sslKeysfile: deprecated; use 'clientCert' instead
11-22-2017 18:09:35.434 -0800 WARN SSLOptions - authentication.conf/[saml]/sslKeysfilePassword: deprecated; use 'sslPassword' instead

We modified as instructed, but every time we had to make a change in the GUI, it changed the suggested parameters back to the depreciated ones. We finally got it working, but it was a major PITA.

More recently we are using a Universal Forwarder install on a dedicated Win2016 VM to ingest Windows Forwarded Events. GPO configures PCs to forward events to Win2016 VM running Universal Forwarder. The events are forwarded to Splunk but they always go to the "main" index. We've tried everything to get them to go to a dedicated index like our other numerous SYSLOG sources. It is very frustrating and I suspect this might be a related issue where despite us seeing the Win2016 host as a new data source, selecting Forwarded Events, and choosing a new index, those settings don't get updated in the proper conf file (we aren't sure where that is for the UF inputs). This is despite the GUI showing all indications that is how it is configured.

We have twice removed UF and all configs on Splunk and the source server and tried to recreate to no avail. Splunk GUI shows that data source is associated with index "wef" but searching index=wef shows nothing. Searching index=main shows the events.

As a last result we tried to manually update the inputs.conf file but that didn't work after a Splunkd restart.

We are opening a ticket, but super frustrating.

[dfollis]

@ravitejaj

I think this is more of a log cluttering issue than performance impacting. Not sure if you are running Splunk on nix or Windows. We are running it on Win Svr 2016. The fix is to modify those fields in the authentication.conf file in /local. Make sure you edit the correct one as there are multiple copies of it in multiple locations.

The odd behavior is that if you edit the file directly to the proper parameter names and then attempt to make a change in the GUI, it modifies the .conf file back to the depreciated parameters. You should be able to modify the .conf file directly and then restart Splunk to see if it works.

Are you using OKTA? We found the following to be most helpful.

1. Use Splunk to monitor "index=_*" real-time when making authentication attempts to see which errors are being generated.
2. Use a SAML debugger such as https://chrome.google.com/webstore/detail/saml-message-decoder/mpabchoaimgbdbbjjieoaeiibojelbhm?hl=e... with Chrome. This gives you insight as to why the attempt might be failing.

In our case with OKTA, using the above tool allowed us to identify that we had the IDP string mis-configured and what Splunk was expecting was not what OKTA was providing.

[ravitejaj]

Can you please let me know, how you were able to manage the below errors? I'm currently stuck up with the same.

11-22-2017 18:09:35.434 -0800 WARN SSLOptions - authentication.conf/[saml]/sslKeysfile: deprecated; use 'clientCert' instead
11-22-2017 18:09:35.434 -0800 WARN SSLOptions - authentication.conf/[saml]/sslKeysfilePassword: deprecated; use 'sslPassword' instead

[hardikJsheth]

Is this standalone environment or clustered environment?

[dfollis]

Stand Alone