Monitoring Splunk

How to search Splunk's internal audit events to see which users are logging in and logging out?

jravida
Communicator

Hi folks,

Been doing a bit of digging within Splunk to see who is logging in and out. I don't see a clear event in the audit.log, when I search:

index = _audit | audit 

I see a lot of other activity, like created, edit_roles, search, quota, but nothing that clearly says logon/logoff.

Am I looking in a weird place, or is it in there somewhere?

0 Karma

deepamshah
Explorer

What works for us. We have a SHD cluster and authentication is provided by ADFS SSO.

For every query, add quotes around action=log* .. For some reason, splunk is NOT detecting action = login attempt as a key-value pair ... may be because there is a space ? ...

Example:

index=_audit sourcetype=audit user=* "action=log*"

The above works!

0 Karma

masonmorales
Influencer

Going for the accepted answer here...

index=_audit sourcetype=audittrail user=* action=log* | rename info as status | replace succeeded with success in status | replace failed with failure in status | replace "login attempt" with login in action |  stats count by user action status  | append [search index=_internal sourcetype=splunk_web_service user=* action=log* | stats count by user action status] | join user [search index=_internal sourcetype=splunk_web_service | stats first(clientip) as clientip by user] | table user clientip action status count

Or with timestamps...

index=_audit sourcetype=audittrail user=* action=log* | rename info as status | replace succeeded with success in status | replace failed with failure in status | replace "login attempt" with login in action |  stats count by user action status _time | append [search index=_internal sourcetype=splunk_web_service user=* action=log* | stats count by user action status _time] | join user [search index=_internal sourcetype=splunk_web_service | stats first(clientip) as clientip by user] | table _time user clientip action status count

jblaine
Explorer

Works for me. Using Splunk + LDAP auth.

EDIT: Actually, I should clarify. This returns events (example below). It does not render our data in the table.

Audit:[timestamp=05-04-2016 11:15:38.643, user=jblaine, action=login attempt, info=succeeded][n/a]
0 Karma

MuS
Legend

Side note: this will only work if you use Splunk user authentication. If you're on LDAP or SSO it will not work

masonmorales
Influencer

I tested in an Active Directory environment and it still showed the login/logouts of AD users. Not sure about SSO though. Have you tested?

0 Karma

reed_kelly
Contributor

What version of Splunk search head are you using. I am seeing different behavior from one version to the next.

0 Karma

reed_kelly
Contributor

On my 6.2.1 search head, the following is the best I could do. It needs a bit of cleaning up, but the gist is there:

(eventtype=splunk_access tag=authentication) OR (((index=_internal UiAuth) OR (index=_internal sourcetype=splunk_web_service)) action=login OR action=logout) user=* | table _time, action, user, clientip, status

I can't seem to find the client ip for any successful commands. That worries me. Once a user does searches, the client ip does get logged, however. I haven't yet checked to see if this captures SDK connections as well.

0 Karma

reed_kelly
Contributor

Thanks for calling me out. I now see that I assumed the Splunk_SA_CIM app to be installed. I was also trying to get the client_ip in cases where it was missing. I agree that @masonmorales gave a more comprehensive answer, but I was attempting to capture more of the clientip values. Tweaking the search from masonmorales results in more logins and additional clientip values:

index=_audit sourcetype=audittrail user=\* action=log\* | rename info as status | replace succeeded with success in status | replace failed with failure in status | replace "login attempt" with login in action |  stats count by user action status _time | append [search index=_internal sourcetype=splunk_web_service user=\* action=log\* | stats count by user action status _time] | join type=OUTER user [search index=_internal (component=UiAuth OR sourcetype=splunk_web_service) user=\* clientip=\* | stats first(clientip) as clientip by user] | table _time user host clientip action status count
0 Karma

jblaine
Explorer

This doesn't work for me. Same version, 6.2.1. I get zero results where @masonmorales ' answer seems to work. We're using both Splunk + LDAP auth.

0 Karma

masonmorales
Influencer

Try this:

(index=_audit sourcetype=audittrail) OR (index=_internal sourcetype=splunk_web_service) user=* action=login OR action=logout | table _time, action, user, status

If action isn't extracted....

(index=_audit sourcetype=audittrail) OR (index=_internal sourcetype=splunk_web_service) user=* | rex "action=(?<newaction>\w+)" | search newaction=login OR action=logout | table _time, newaction, user, status
0 Karma

jravida
Communicator

After doing the above search (and some bits of it) am able to produce results, but with HUGE gaps. Since I'm not the only engineer on Splunk where I work, I am guessing someone tuned the loggin levels somewhere and, well, as you know people like to go around changing things without documenting it.....
I'm going to meet with my other devs tomorrow and try to figure out what the deal is before I wrack my brain further. Thanks for you help so far!

0 Karma

masonmorales
Influencer

Since this helped you, could you please click "Accept Answer"?

0 Karma

reed_kelly
Contributor

Did you test your answer on Splunk 6.2+ in a distributed environment? I think the answer varies by version.

0 Karma

jravida
Communicator

I just tried this and got 0 results....

0 Karma

masonmorales
Influencer

This does work. Are you running the search on a user that has access to internal indexes (e.g. admin), and in an App context that has access to the internal indexes (e.g. Search)? If yes, try a longer time range.

0 Karma

masonmorales
Influencer

I've removed "source=" from my original post, as that also could have been the issue.

0 Karma

jravida
Communicator

when I look at the sourcetypes and search for login or logout, there is no value of login/logout for action=.
Theres a lot of other stuff for action=, but the login/logout isn't there.

If I dig deep > 1 month, some events do come up related to admin (of which my account is a member) but only a handful. But the action field doesn't seem to be parsing correctly.

If I search for a month long stats list of the action field, I am met with lots of values, like search, quota, edit_user, created, deleted...it goes on and on, but no login or logout.

0 Karma

masonmorales
Influencer

This is probably a stupid question, but you are running this search on the Search Head and not the Indexer, right?

0 Karma

jravida
Communicator

correct, my indexer is on a separate server, I'm doing this on the search head

0 Karma

masonmorales
Influencer

Losing fields in the comments, will add it at the top.

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...