topic How to parse wtmp file in Splunk Search

How to parse wtmp file

jwalzerpitt — Fri, 19 Dec 2014 19:36:36 GMT

I am ingesting the non-binary wtmp file in Splunk and was able to two generic fields: 1) priority = auth. (4 unique), and 2) source IP.

I then broke down each priority to view the unique messages inside each one and identified the relevant messages I'd like to extract fields from. Unfortunately, the format of the file loses its consistency after the ID # auth., where the message starts. What I'd like to extract is the username, but the problem is username is not in the same place in the message for each priority. With that I don't believe there is a generic regex that will capture username, but would to be proven wrong!

So, digging in more, I flagged the following messages which I'd like to extract the username from (and username always follows "for"):

Accepted keyboard-interactive for
Accepted password for
Failed keyboard-interactive for
Failed gssapi-with-mic for
Failed password for
Failed keyboard-interactive for
Failed gssapi-with-mic for
Failed publickey for
Failed password for

Moving forward, was trying to figure out the best way to move forward on this. Do I create separate sourcetypes for these specific logs (was going to look to reverse engineer the Symantec for Spunk app as there are multiple sourcetypes defined - I have my SEP logs funneling through the app and it does a good job breaking the logs out, but Symantec's log format is comma deliminated, making it easier), or do I try and create multiple regexes for one sourcetype (is this even possible?)?

Any ideas would be greatly appreciated.

Thx

Re: How to parse wtmp file

richgalloway — Fri, 19 Dec 2014 19:41:54 GMT

It would help to see the raw data, but this should get you started...

rex field=priority " for (?<username>.*?)"

Re: How to parse wtmp file

jwalzerpitt — Mon, 28 Sep 2020 18:29:12 GMT

Rich,

Thx for the reply.

1) Tried the rex and it's not returning the username field
2) Unfortunately, the username doesn't always follow "for" (which adds to the frustration)
3) Sample info below:

Dec 19 14:14:27 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:27 sshd[5977]: [ID 649047 auth.info] AFS Ignoring superuser root
Dec 19 14:14:27 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:28 sshd[5977]: [ID 800047 auth.notice] Failed password for root from x.x.x.x port 1055 ssh2
Dec 19 14:14:28 sshd[5977]: [ID 800047 auth.info] Disconnecting: Too many authentication failures for root
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] Illegal user admin from x.x.x.x
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] input_userauth_request: illegal user admin
Dec 19 14:14:28 sshd[5980]: [ID 800047 auth.info] Failed none for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5983]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5984]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5985]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:29 sshd[5986]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:29 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5987]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5988]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Failed password for from x.x.x.x port 1188 ssh2
Dec 19 14:14:30 sshd[5980]: [ID 800047 auth.info] Disconnecting: Too many authentication failures for admin
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Illegal user admin from x.x.x.x
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] input_userauth_request: illegal user admin
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed none for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5992]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5993]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:33 sshd[5994]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:33 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5995]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5996]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:34 sshd[5997]: [ID 818691 auth.error] AFS Authentication failed for user admin. user doesn't exist
Dec 19 14:14:34 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2
Dec 19 14:14:35 sshd[5989]: [ID 800047 auth.info] Failed password for from x.x.x.x port 2441 ssh2

Re: How to parse wtmp file

richgalloway — Fri, 19 Dec 2014 20:25:28 GMT

How about this?

 rex field=priority "(?:for|user|superuser) (?<username>\S+)"

It will return "from" as the user name in the "Failed password" events, however, since that event seem to have no user name in them.

Re: How to parse wtmp file

jwalzerpitt — Fri, 19 Dec 2014 20:36:32 GMT

Improved search when I drop the field=priority. I am seeing unique usernames, but also getting some non-usernames, such as:

Timeout before authentication for x.x.x.x
the word "for"
log files other than the ones listed in original message:

Accepted keyboard-interactive for
Accepted password for
Failed keyboard-interactive for

That's where the frustration comes in as the regex is getting a majority of the valid usernames, but it's still grabbing some values that aren't usernames.

And for edification on my part, the ?: is a non-capturing subpattern that is looking for "for|user|superuser" as a starting point, and then matches on everything after up untila nd including the space, correct?

Thx again