I can't connect Splunk SOAR to Splunk Enterprise Security

[SplkhdA_1]

First of all, hello everyone. I have a mac computer. I installed Splunk enterprise security on this Mac M1 computer. Then I wanted to install Splunk SOAR, but I could not install it due to centos/RHEL arm incompatibility installed on the virtual machine. Then I rented a virtual machine from azure and installed Splunk SOAR there. Splunk enterprise is installed on my local network. First, I connected Splunk Enterprise to SOAR by following the instructions in this video (https://www.youtube.com/watch?v=36RjwmJ_Ee4&list=PLFF93FRoUwXH_7yitxQiSUhJlZE7Ybmfu&index=2) and test connectivity gave successful results. Then I tried to connect SOAR to Splunk Enterprise by following the instructions in this video (https://www.youtube.com/watch?v=phxiwtfFsEA&list=PLFF93FRoUwXH_7yitxQiSUhJlZE7Ybmfu&index=3), but I had trouble connecting soar to Splunk because Splunk SOAR and Splunk Enterprise Security are on different networks. In the most common example I came across, SOAR and Splunk Enterprise Security are on the same network, but they are on different networks. What should I write to the host ip here when trying to connect SOAR? What is the solution? Thanks for your help.

[PickleRick]

It's not just "because they are in different networks" but because your internal network is organized the way it is. You might try to set up a VPN to your Azure environment to allow for connectivity or try to do some DNATs in your home network but as you're asking a kind of very basic network-related questions, you'd better not do that without fully understanding the risks.

[SplkhdA_1]

Actually I did what you said. I asked this question to the community to make sure I was doing it right, maybe I was missing something. SOAR is installed on centos 8.5 operating system. I couldn't install openvpn on this OS. I rented another virtual machine and installed openvpn on it. VPN machine and SOAR were on different networks again, I peered them over azure. CentOS 8.5 machine and openvpn machine were on the same network. When I connect to VPN from my computer, I can ping the centOS private IP address from my computer and get a response, there is no problem here. But Splunk SOAR still refuses to connect 😞

[PickleRick]

It actually depends on your network environment and your configuration.

The general answer is - you must have network connectivity between the environments and the proper traffic must be allowed on local OS-level firewall  In this aspect SOAR and Splunk Enterprise are not different from any other network services - you must have an ability to connect to a port to be able to use it as simple as that.

So it's not that I'm trying to be rude or something, it's just that there are so many variables here that it'd be better if you engaged your local network/linux guru to help you because that's something that's not Splunk-specific and local help will be much more responsive than ping-ponging stuff over internet forum.

[SplkhdA_1]

Hello again. I made sure that my centOS computer with Splunk SOAR installed and my MAC computer with Splunk Enterprise installed are on the same network. CentOS is installed on azure. I enabled my Mac computer to access the azure network with Virtual Network Gateway. CentOS and Mac computers can ping each other but I can't access port 8089. Do I need to do something with splunk enterprise for this?

[PickleRick]

Ok. Did you check whether/on which ports is your Splunk server listening?

Did you try to connect to other ports/services on your Mac?

Does your Mac have any kind of host firewall? (I don't know Mac so I have no idea if if it does something like that or not).

These are the basic, not Splunk related as such, things you should check. That's why I suggested you get help locally from someone who knows networking - that would probably be much more quicker than to "remotely diagnose" this issue over such slowly responsive medium.

[SplkhdA_1]

Hello. Splunk can now listen to all ports related to itself.

I have no problems connecting to other services on the mac.

I gave splunk full permission for incoming connections in the firewall on Mac.

Unfortunately, it does not happen despite this. Either I am missing a very simple point or there is a bug in the soar program for centOS 8 and I will wait for the next update :))

[PickleRick]

OK.

I assume you can  connect to Splunk's port 8000 locally, right?

Can you do the same from the SOAR machine? (using curl, for example)?

BTW, do you have SELinux enabled on the SOAR machine?

[SplkhdA_1]

Yes. I can connect to Splunk from SOAR machine.

in linux now firewalld works with splunk phantom. It never occurred to me to check SELinux. It works in Enforcing mode, but I don't understand what exactly is the effect on SOAR. Would it be appropriate to disable it or put it in Permissive mode?

[PickleRick]

You should check auditd logs to see if SELinux prevented your connectivity. SELinux does prevent unauthorized connectivity. Of course for short-term testing you can simply switch SELinux to permissive or disable it. Preferable good-time solution would be to either find a tunable boolean in the policies to allow this if there is one or adjust the policies.

[SplkhdA_1]

I couldn't find any selinux blocking in the audit logs. I disabled it for testing anyway but nothing changed.

[PickleRick]

Ok. So you can connect to Splunk's 8000 port from the SOAR machine, can connect to Splunk's 8089 port from local network and cannot connect to 8089 from SOAR computer? (I'm talking about connection with telnet/netcat/curl/openssl, not from the SOAR itself)

[SplkhdA_1]

yes, I can not connect to 8089 port from Soar machine to splunk enterprise machine with using CLI(telnet/netcat/curl/openssl).  I scanned 8089 port with using nmap and It says refused the connection. I think it might be an issue with the Azure platform. The Virtual Machine(CentOS) might be refusing to connect to the external network and this might be related to azure. I will contact Azure support team. 

[PickleRick]

So you can connect with another tool but not with nmap? Something's fishy here.

EDIT: You wrote "cannot". So if you can connect to 8089 locally and cannot from the soar machine it's something to be resolved on networking level.