Getting Data In
Highlighted

How to run multiple universal forwarders on a single Linux host?

Path Finder

I am trying to install 2 universal forwarders on a single Linux host.
I read a few articles and changed the httpport and mgmtHostPort in the web.conf file in $SPLUNK_HOME/etc/system/local.
I also changed serverName in server.conf file.
On trying to start the new splunk instance, it gives below error:

The splunk daemon (splunkd) is already running.            [FAILED]

Is there something else that needs to be changed or is there a different method of running 2 universal forwarders on 1 host?

Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Influencer

Should be as simple as unzipping the tar in a different directory, and when you start it from that second bin folder it will ask you to change the ports. Seems to me you are trying to start splunk from the instance thats already running.

Do you mind me asking why you would want to do this? There are better options depending on what your trying to do.

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Path Finder

Like I mentioned, we have already changed the ports and the server name but its still not working. The reason why we want to do this is because we want 2 different flavours of universal forwarder on 1 host. Each one will be scanning a different set of directories.

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Communicator

I think you have done the necessary change. You could check the process status (ps) to make sure that there is no previous Splunk running process when you start. Also, after startup the first one, check to see it's using the managment port you set.

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Path Finder

I stopped the forwarder that was running already and then tried to start the new one.
I have modified web.conf in /opt/splunkforwarder2/etc/system/local directory with the below details:
[default]
[settings]
httpport = 5000
mgmtHostPort = 127.0.0.1:8099

When I try to start splunkforwarder2, it is still using mgmt port 8089. Is it possible that its still reading the configuration from /opt/splunkforwarder rather than /opt/splunkforwarder2?

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Path Finder

I have confirmed my findings. Changing mgmt port via CLI in /opt/splunkforwarder2 still makes changes in /opt/splunkforwarder/etc/system/local.web.conf
There must be a way to decouple the 2 directories so that they look at their own config directories.

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Influencer

hmm maybe an environment variable thing?

How did you install? tarball? what command are you running and from which path to change the port?

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Path Finder

I tried exporting $SPLUNK_HOME as /opt/splunkforwarder2 but it didnt make a difference. Changes are still getting done in /opt/splunkforwarder.
Manually changing the config files in /opt/splunkforwarder2/etc/system/local isnt helping either (probably because this instance is somehow reading config from /opt/splunkforwarder/etc/system/local).

Regarding installation, I did the first one through rpm and then copied /opt/splunkforwarder to /opt/splunkforwarder2 to create a second directory containing all the binaries and other sub-directories.

0 Karma
Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Influencer

I think this is where your issue lies. I did my answer below with tar both times. I bet if you do rpm then tar instead of copy it would have worked

Highlighted

Re: How to run multiple universal forwarders on a single Linux host?

Esteemed Legend

AHA! We may have a winner. I never use RPMs and always use the tarballs and it always worked fine for me.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.