topic Re: Strange Splunk Ports in Splunk Search

Strange Splunk Ports

JarrettM — Mon, 30 Apr 2018 21:19:29 GMT

All 37 of my Splunk forwarders establish TLS 1.2 connections to Splunk on port 9997 as configured. No problem there. But splunkd is also listening on 8884 and 6 of the forwarders continually attempt a raw connection to this port in addition to the TLS one they make on 9997. The connections are not established but still I would like to close the port on both sides. The configurations on those 6 are the same as on the other 31 that are that using not using that port. Netstat confirms that it is splunkd that is listening on 8884.

Any ideas why splunkd would be listening on that port, why those 6 forwarders are attempting to connect on it and how to close it on both sides?

Thanks!

Re: Strange Splunk Ports

JarrettM — Mon, 30 Apr 2018 21:22:48 GMT

Windows 2012 R2 Server

Re: Strange Splunk Ports

xpac — Mon, 30 Apr 2018 22:00:48 GMT

Port 8884 isn't any default port used by Splunk in any way. Just to make sure I searched the docs and Google for it, but there is absolutely zero about it (as you might already have noticed).
I'd search through all .conf files on the affected servers for the string "8884" - it must be mentioned anywhere.
Did you, by any chance, install a certain add-on or app that might have opened that port?
You could also do this from the CLI: splunk btool inputs list and check the output for 8884 anywhere.

Besides that, I'm a little out of ideas on this.

Re: Strange Splunk Ports

JarrettM — Tue, 01 May 2018 12:26:46 GMT

Thanks! Ran that CLI and found it. It's an input from a syslog that I forgot to check.

Thanks again!

Re: Strange Splunk Ports

amitm05 — Tue, 01 May 2018 12:49:44 GMT

This is a strange port in context to splunk.
However I'd do the following :

Verify my Web and Management ports on all the indexing machines by running the commands - ./splunk show web-port ./splunk show splunkd-port

And also the receiving port.

This will take you one step forward in troubleshooting, either you'd find that these are the default ones i.e. 8000, 8089 and 9997.
OR
one of the services is using 8884 or may be the multiple receiving ports (i.e. 9997 and 8884) have opened.

And if none of the above happens and no service shows 8884 port, I would use the netstats command to find out the PID which is using 8884 and Kill it (Assuming its a bogus process showing up by the name of splunkd). OR as @xpac says - search through all .conf files on the affected servers for the string "8884"

I hope this may take you closer to your solution !