Monitoring Splunk

Missing Audit logs

RDAVISS
Path Finder

I just noticed that our Redhat splunk servers are missing audit log data for users logging in to Splunk.

For example, this query no longer returns data:
index=_audit action="login attempt" "info=succeeded"

I do have some audit data, just not the login attempts.

The data seems to of stopped after upgrading to version >=8.0.0

I only have one windows splunk server, and ALL the audit data appears to be there.

Labels (1)
0 Karma

RDAVISS
Path Finder

I engaged splunk support and there was a code change around version 8.x that might have caused this to stop working.

Once I have a workaround I will post it here.

0 Karma

codebuilder
Motivator

Sounds like your search heads are no longer forwarding internal logs to the indexer cluster.
Ensure they are configured to do so by examining $SPLUNK_HOME/etc/system/local/outputs.conf to verify the SHC is sending those logs to the indexers. And/or look at inputs.conf to verify there are no blacklists that might be blocking those logs.

https://docs.splunk.com/Documentation/Splunk/8.0.2/Admin/Outputsconf#outputs.conf.example

0 Karma

RDAVISS
Path Finder

We double-checked the outputs and don't see any errors. The indexer is getting some events (e.g failed logins are showing up, just not the successful logins. )

I will take a look at the inputs again just to make sure there are no problems with our blacklisting.

What's strange is the successful events aren't actually on the local search heads logs
(/$SPLUNK_HOME/var/splunk/log/audit.log)

That's why I leaning towards a change in functionality with the 8.0 release.

CyberWarrior404
New Member

Did you manage to find the solution as to why the login activity is not showing up in Splunk 8.X?

0 Karma

soutamo
SplunkTrust
SplunkTrust

Hi

at least all in one node with 8.2.0 the event is still the same

Audit:[timestamp=05-29-2021 23:06:55.778, user=XXXXX, action=login attempt, info=succeeded reason=user-initiated useragent="Mozilla/5.0 (XXX; XXX ) XXXXX/....." clientip=127.0.0.1" method=Splunk" session=ecd9cadalsdklakdlakd8d5391718ea8]

I haven't access to distributed environment to check it (only 8.x.x).

index = _audit sourcetype=audittrail action="login attempt" info=*
| stats count by user,info

Previous example founds users as expected.

r. Ismo 

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!