topic Re: Failed Logon attempt Correlation in Splunk Search

Failed Logon attempt Correlation

pstephens93 — Tue, 05 Jan 2021 15:10:29 GMT

Hello,

Our environment has this linux server that continually get's hit with Brute force attacks. I am trying to figure out where they are coming from. Since our servers are behind a nated firewall I need to be able to see the failed logon attempts and match the server IP , port number to the DestinationIP and natSourcePort

I am trying to use sub search. I have one search that searches all of our indexes for failed passwords from server IP and I want it to return the portnumber

Then do a second search that matches the IP and Port numbers. of the first search and return the top Source addresses

Here is what I have so far. Any help would be apricated

index="palo" ( PA_natDestinationIP=Server IP ANDPortNumber=PA_natSource_Port )
[ search index= "*" Failed password IP = ServerIP | return PortNumber ]
| stats count by PA_SourceAddress |sort by count desc

Re: Failed Logon attempt Correlation

richgalloway — Tue, 05 Jan 2021 15:46:03 GMT

What is your question? You have a search, so what's the problem? What help is needed?

You use "index=*" in a production query so no pudding for you tonight. Take some time to figure out which indexes contain failed password events and have the query search only those indexes. Your indexers will thank you for it.

Re: Failed Logon attempt Correlation

pstephens93 — Tue, 05 Jan 2021 16:37:41 GMT

So I have my two searches,
The inner search finds the failed password attempts for a particular server ( I tested it and it works) I need this to return a list of all the Port numbers used for the failed logon attempts . The return portion does not seem to work

The outer search will seach for Server IP(DestionationIP) and take the list of port numbers returned from the first search and match it with the nat port number.

I am having trouble with the return portion of the search s well as with the out search. I am not sure if the return function is not returning the list so it can not match up port numbers.

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port )
[ search index="wawf" (Failed password AND Linux_Server="server1") | return empherical_Port ]
| stats count by PA_SourceAddress |sort by count desc | Table SourceAddress, Empherical_Port, NatSource_Port

Re: Failed Logon attempt Correlation

richgalloway — Tue, 05 Jan 2021 16:57:28 GMT

The return command returns a single value by default.

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port ) [ search index="wawf" (Failed password AND Linux_Server="server1") | return 1000 empherical_Port ] | stats count by PA_SourceAddress | sort by count desc | Table SourceAddress, Empherical_Port, NatSource_Port

Run the subsearch by itself to see what it will return to the outer search.

Re: Failed Logon attempt Correlation

pstephens93 — Tue, 05 Jan 2021 19:38:50 GMT

When I ran the sub Search on it's own I got a values like (ephemerical_Port="63450") I added a $ to the fieldname and it dropped the Empherical_Port and updated the results to and example like (63450)

I think the problem now is it is trying to look through and match up

Empherical_Port=natSource_Port

(63450) = 63450 which would show as not a match.

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port ) [ search index="wawf" (Failed password AND Linux_Server="server1") | return 1000 $empherical_Port ] | stats count by PA_SourceAddress | sort by count desc | Table SourceAddress, Empherical_Port, NatSource_Port

Re: Failed Logon attempt Correlation

richgalloway — Tue, 05 Jan 2021 21:49:50 GMT

What happens with subsearches is the results replace the subsearch in the outer search. In our case, the search would become

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port ) (63450) | stats count by PA_SourceAddress | sort by count desc | Table SourceAddress, Empherical_Port, NatSource_Port

This is valid, but inefficient (Splunk will look in all fields for "63450") and may return unexpected results (63450 may be something other than a port number).

The solution is to make sure the subsearch returns a field name known to the outer search.

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port ) [ search index="wawf" (Failed password AND Linux_Server="server1") | rename empherical_Port as Empherical_Port | return 1000 Empherical_Port ] | stats count by PA_SourceAddress | sort by count desc | Table SourceAddress, Empherical_Port, NatSource_Port

Please confirm Empherical_Port and natSource_Port are both present in the events. If they are not both present then the search will find nothing.

Re: Failed Logon attempt Correlation

pstephens93 — Mon, 11 Jan 2021 17:33:28 GMT

I am still having issues with this. The sub search returns the following:

When I run the full search together I get nothing, When I just run the outer search with hard coded values
I get a match. When I do Empherical_Port = 20961 which is in the port values list about 200 values into the list. When the searches are combined I get nothing, when I do the inner search then hard code the values into the outer search. Some get hits some do not. Is it not running through the whole return list?

index="palo" ( natDestinationIP=10.63.16.103 AND Empherical_Port=natSource_Port ) index="palo" ( natDestinationIP = 10.63.16.103 AND natSourcePort =20961 )

Re: Failed Logon attempt Correlation

richgalloway — Mon, 11 Jan 2021 18:07:54 GMT

The subsearch is returning a field called "Port". Does the outer search have such a field? I suspect it does not since you use Empherical_Port in your example. The subsearch must return a field name known to the outer search to produce results.

Re: Failed Logon attempt Correlation

pstephens93 — Mon, 11 Jan 2021 18:51:38 GMT

The Inner search I renamed the Field ephemeral_Port to Port.

When I break down the Searches into the following:

index="wawf-prod" (Failed password AND Linux_Server="wawfprodftpd37") |rename ephemeral_Port as Port |return 1000 Port

This Returns the list of Ports as shown in the above screen shot I sent.

When I run the Code below with a hard coded value for the PA_natSourcePort = 20961 It returns a match. the port number 20961 is one of the values returned from the first search that I hard coded to see if It was a values issue. Which some of the values hard code match and some do not.

index="palo-test" PA_natDestinationIP=10.63.16.103 AND PA_natSourcePort = 20961

So Something is happening in the PA_natSourcePort = port when the two searches are combined with a sub search. Any ideas why that is?

index="palo-test" ( PA_natDestinationIP=10.63.16.103 AND PA_natSourcePort = Port) [ search index="wawf-prod" (Failed password AND Linux_Server="wawfprodftpd37") |rename ephemeral_Port as Port |return 1000 Port ] | stats count by PA_SourceAddress |sort by count desc | table PA_SourceAddress, Port, PA_natSourcePort, count

Re: Failed Logon attempt Correlation

richgalloway — Mon, 11 Jan 2021 22:00:04 GMT

At the risk of repeating myself, the subsearch must return field names that appear in index=wawf-prod. If it doesn't then the main search will find no results.