Why is my real-time search correlation for SSH detecting false positives with user sshd?

reljssplunk — Fri, 24 Apr 2015 19:13:56 GMT

After searching for a bit, I can't find an exact fix to this issue--
I'm having some weird edge cases with a realtime search that we're using to alert on SSH logins.

Under normal circumstances the search below triggers an alert if there is a successful login by a user after 4 failed logins in 5 minutes. Normally this works fine, however yesterday we showed brute force attempts from someone trying to log in as a user called "sshd".

This caused the search to trigger for any users successfully logging in within 5 minutes of the attempt, I'm assuming because it interpeted their successful authentication attempts as successes by the "sshd" user.

Is there any good way to modify my search to account for this?

Search code:

ssh* authenticated [search ssh* "authentication failure" | rex field=_raw "rhost=(?<ipfrom>.*) user=(?<username>.*)" | stats count by username | where count >= 4 | rename username AS query | fields query ]

Sanitized logs:

Apr 23 14:15:55 hostX auth.debug<39>: sshd[12243]: debug1: monitor_child_preauth: bob has been authenticated by privileged process
Apr 23 14:15:53 hostX authpriv.info<86>: sshd[12245]: pam_krb5(sshd:auth): user bob authenticated as bob@myhost.net
Apr 23 14:12:27 hostX authpriv.notice<85>: sshd[15524]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=100.100.100.555 user=sshd
Apr 23 14:12:13 hostX authpriv.notice<85>: sshd[15524]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=100.100.100.555 user=sshd
Apr 23 14:11:44 hostX authpriv.notice<85>: sshd[15440]: pam_sss(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=100.100.100.555 user=sshd
Apr 23 14:10:32 hostX authpriv.notice<85>: sshd[15440]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=100.100.100.555 user=sshd

Re: Why is my real-time search correlation for SSH detecting false positives with user sshd?

aweitzman — Fri, 24 Apr 2015 20:46:52 GMT

Your problem is that the sshd returned from your subsearch appears to be matching the sshd in the name of the process, not the user name.

To fix this, you need to rex your event into fields, assign the username to a field name, and then at the end of your subsearch, assign the result to that field name rather than query. That ought to get you what you want.

topic Re: Why is my real-time search correlation for SSH detecting false positives with user sshd? in Alerting

Why is my real-time search correlation for SSH detecting false positives with user sshd?

Re: Why is my real-time search correlation for SSH detecting false positives with user sshd?