How to install a heavy forwarder and a search head on the same VM?

[antoniorp77]

Hi,

I have installed a Splunk instance that serves as a search head and now I need to install another instance to serves as a heavy forwarder. However when I download the Splunk file and extract it on a different directory than my first instance it tells me that port 8000 is being used and then it asks me to give it different ports since other daemon ports are being used as well. Is this normal? Is this the standard procedure to do this? I just need both instances to be running on port 8000 on the same VM. Also I need to ssh into my search head instance however when I run ssh [hostname]@[private-ip:8000] I get an error saying "could not resolve hostname". I would really appreciate some guidance. Thanks.

[PickleRick]

Why would you do something like that?

If you have spare computing power, just spin up a new VM and deploy a separate instance there.

[antoniorp77]

If I hit "n" then it just exits the installation.

[gcusello]

Hi @antoniorp77,

as @PickleRick said, why do you want to do this?

apparently, there isn't any sense to this configuration because if you need of a system that concentrates logs or takes syslogs, you can use the same SH instance or one Indexer.

In addition, two instances require double resources: have hou 28 CPUs and 24 GB RAM on your Splunk server?

At least, if you have these resources and you want to have this non sense configuration, you can use different ports to access both the instances and they separately run: I tried (only in test!) to have different instances on the same server and correctly runned ussing diferent ports.

Ciao.

Giuseppe

[antoniorp77]

Hi @gcusello thank you for your response.

I was trying to install two different instances on the same server because I am taking a Splunk course and that is how the teacher did it. However, he did not show how he did it. All I was able to see was that he had two different Splunk instances on the same browser, on the same VM with different IP addresses and different apps running on the same port (8000). He configured one of them to be a search head and the other one to be a heavy forwarder. From what you are telling me, I assume that I should configure the heavy forwarder on the same instance where I configured my search head?

[PickleRick · Accepted solution]

Strictly theoreticaly - you can install many instances on one OS (physical computer or VM). But you should not do that. In some lab situations it can be used to limit used resources (not need to spin up a new VM for each component) but it's a very tricky setup, it's much more complicated than installing normally - one component per VM - and it's a completely unsupported installation. And there's so much that can go wrong.

Simply put - don't do it like that. One splunk instance per VM is the way you should go.

Also - you should not mix roles on one VM. There are some acceptable exceptions to this rule in smaller installations but you should not mix HF and SH functionalities.

So if you need SH and HF, you should have a separate VM for SH and separate VM for HF.

[antoniorp77]

Hi @PickleRick,

Thank you for explaining. It has worked now that I have installed the SH on one VM and the HF on another on another VM.

[JacekF]

To avoid multiple VMs, you can also use Splunk in docker splunk/splunk - Docker Image | Docker Hub

[PickleRick]

To be fully honest, I could never grasp the idea behind dockerized splunk. In order to have persistent configuration and data (and splunk doesn't make sense otherwise) you still have to externalize the important stuff from the docker container so in fact you're simply pointlessly doing a deployment from scratch every time you launch your container.

I know that containers are "the thing" nowadays but they are relatively good for thin stateless services, not for a quite huge - in comparison - solution that splunk is.

[antoniorp77]

@PickleRick

It does solve the problem, however one thing concerns me. If I have multiple VMs running on Kali Linux each of them will take at least 8GB to install. So do you know if there is a way to do this without consuming so much space? Considering that I will be only using the extra VMs just as Splunk instances.

[PickleRick]

Well, if it's a production environment, you have your minimal recommended component sizes and there's really no way around it (in fact 8GB for a production search-head is a very low-spec machine; you should have at least 16GB).

But if it's a lab one - well, you can try to install it on a lower-spec machine but be prepared that it might crash. The HF should not, however have that huge memory footprint since it shouldn't do very memory-intensive operations - it "just" parses the data and sends them out.