Monitoring Splunk

Missing Audit logs

Path Finder

I just noticed that our Redhat splunk servers are missing audit log data for users logging in to Splunk.

For example, this query no longer returns data:
index=_audit action="login attempt" "info=succeeded"

I do have some audit data, just not the login attempts.

The data seems to of stopped after upgrading to version >=8.0.0

I only have one windows splunk server, and ALL the audit data appears to be there.

Labels (1)
0 Karma

Path Finder

I engaged splunk support and there was a code change around version 8.x that might have caused this to stop working.

Once I have a workaround I will post it here.

0 Karma


Sounds like your search heads are no longer forwarding internal logs to the indexer cluster.
Ensure they are configured to do so by examining $SPLUNK_HOME/etc/system/local/outputs.conf to verify the SHC is sending those logs to the indexers. And/or look at inputs.conf to verify there are no blacklists that might be blocking those logs.

0 Karma

Path Finder

We double-checked the outputs and don't see any errors. The indexer is getting some events (e.g failed logins are showing up, just not the successful logins. )

I will take a look at the inputs again just to make sure there are no problems with our blacklisting.

What's strange is the successful events aren't actually on the local search heads logs

That's why I leaning towards a change in functionality with the 8.0 release.

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!